-- Logs begin at Tue 2022-07-12 12:46:30 UTC, end at Tue 2022-07-12 13:42:32 UTC. -- Jul 12 13:12:54.791861 dv-cinder-820027-10-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 12 13:12:54.799752 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Jul 12 13:12:54 2022] *** Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: nodename: dv-cinder-820027-10-iscsi Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: machine: x86_64 Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: clock source: unix Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: pcre jit disabled Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: detected number of CPU cores: 4 Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: current working directory: / Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: detected binary path: /usr/bin/uwsgi-core Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: your processes number limit is 47680 Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: your memory page size is 4096 bytes Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: detected max file descriptor number: 2048 Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: lock engine: pthread robust mutexes Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: thunder lock: enabled Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 12 13:12:54.803780 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 12 13:12:54.847687 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: Python main interpreter initialized at 0x55e3b73bc9c0 Jul 12 13:12:54.847687 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: python threads support enabled Jul 12 13:12:54.847687 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: your server socket listen backlog is limited to 100 connections Jul 12 13:12:54.847687 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: your mercy for graceful operations on workers is 90 seconds Jul 12 13:12:54.848216 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: mapped 403077 bytes (393 KB) for 2 cores Jul 12 13:12:54.848385 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: *** Operational MODE: preforking *** Jul 12 13:12:54.848814 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: *** uWSGI is running in multiple interpreter mode *** Jul 12 13:12:54.848814 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: spawned uWSGI master process (pid: 80277) Jul 12 13:12:54.849036 dv-cinder-820027-10-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 12 13:12:54.853398 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: spawned uWSGI worker 1 (pid: 80287, cores: 1) Jul 12 13:12:54.853398 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: spawned uWSGI worker 2 (pid: 80288, cores: 1) Jul 12 13:12:54.853398 dv-cinder-820027-10-iscsi devstack@keystone.service[80277]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 12 13:13:00.304319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 13:13:00.304319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: from cryptography.utils import int_from_bytes Jul 12 13:13:00.305008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 13:13:00.305008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: from cryptography.utils import int_from_bytes Jul 12 13:13:00.305008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 13:13:00.305008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: from cryptography.utils import int_from_bytes Jul 12 13:13:00.305600 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 12 13:13:00.305600 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: from cryptography.utils import int_from_bytes Jul 12 13:13:00.354340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 13:13:00.354550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 13:13:00.354766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] command line args: [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 13:13:00.354867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 12 13:13:00.354983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 12 13:13:00.355099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 13:13:00.355204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] command line args: [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 12 13:13:00.355283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ================================================================================ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 13:13:00.355504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 12 13:13:00.355588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] admin_token = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.355665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ================================================================================ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 12 13:13:00.355749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] config_dir = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.355920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.356003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] admin_token = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.356100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] config_source = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.356183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config_dir = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.356279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] debug = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.356368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.356476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.356783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] config_source = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.356783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] debug = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.356987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.357091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.357214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.357311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.357311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.357645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.357751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.357849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.357849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_config_append = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_config_append = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358553 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_dir = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_file = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_dir = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.358855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.359072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.359180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.359294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.359407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.359517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.359630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.359747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.359747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.359949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.360064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.360187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.360187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.360417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.360537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.360657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.360779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.360894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.361546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.361710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.361884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.362024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.362153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.362252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.362348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.362451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.362571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.362691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.362850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.362961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.363081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.363197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.363197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] publish_errors = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.363418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.363533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.363635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] publish_errors = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.363757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.363870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.363989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.364108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.364221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.364338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.364469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.364595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.364708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] standard_threads = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.364820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.364932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.365095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] standard_threads = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.365214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.365328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.365478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.365601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.365721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_journal = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.365843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.365963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_json = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.366076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_journal = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.366195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_stderr = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.366308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_json = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.366427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] use_syslog = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.366549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_stderr = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.366664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.366777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] use_syslog = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.366895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.367009 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 12 13:13:00.367137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.367258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.367372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.367485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.367596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.367710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.367833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.367947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.368060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.368178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.368291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.368403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.368522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.368637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.368757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.368877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.368990 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.369102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.369223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.369343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.369484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.369605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.369718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.369843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.369969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.370084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.370201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.370320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.370440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.370553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.370743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.370867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.370981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.371094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.371206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.371317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.371437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.371552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.371670 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.371789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.371903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.372023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.372023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.372244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.372356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.372468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.372581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.372693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.372817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.372933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.373045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.373158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.373269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.373411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.373541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.373657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.373769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.373881 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.374000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.374126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.374247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.374367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.374479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.374591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.374703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.374829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.374944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.375058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.375170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.375286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.375399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.375520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.375634 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.375746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.375858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.375977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.376097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.376285 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.376514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.376681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.377315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.377480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.377595 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.377694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.377796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.377916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378031 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.378925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.379108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.379206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.379301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.379395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.379498 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.379603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.379698 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.379853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.379968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.380187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.380323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.380439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.380559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.380664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.380776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.380894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.381007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.381127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.381273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.381417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.381528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.381641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.381768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.381892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.382008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.382120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.382242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.382358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.382526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.382656 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.382780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.382896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.383009 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.383122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.383236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.383355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.383478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.383593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.383706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.383809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.383921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.384034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.384156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.384271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.384385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.384499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.384613 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.384726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.384847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.384962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.385076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.385188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.385304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.385474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.385599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.385715 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.385829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.385942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.386056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.386180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.386296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.386410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.386525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.386638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.386752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.386878 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.386999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.387114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.387228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.387341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.387454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.387577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.387692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.387812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.387927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.388040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.388160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.388350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.388468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.388582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.388697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.388817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.388932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.389053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.389177 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.389307 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.389447 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.389564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.389679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.389793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.389908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.390033 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.390149 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.390262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.390376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.390490 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.390603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.390726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.390841 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.390955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.391069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.391182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.391296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.391417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.391542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.391657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.391773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.391886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.392000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.392123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.external = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.392296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.392457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.392559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.external = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.392680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.392790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.392914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.392988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.password = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] auth.token = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.password = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] auth.token = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393841 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.393994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.394918 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.395034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.395148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.395263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.395394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.395511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.395634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.395750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.395864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.395986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.396102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.396216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.396339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.396455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.396569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.396684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.396798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.396913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.397037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.397154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.397269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.397397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.397514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.397629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.397755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.397755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.397972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.398087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.398201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.398315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.398438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.398438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.398667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.398782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.398896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.399010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.399133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.399133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.399349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.399349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.399570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.399685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.399807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.399923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.400036 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.400152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.400266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.400380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.400502 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.400502 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.400724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.400724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.400947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.401070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.401262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.401421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.401540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.401655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.401769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.401884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.402010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.402127 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.402243 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.402363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.402479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.402606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.402724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.402839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.402953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.403068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.403188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.403312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.403429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.403429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.403644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.403757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.403878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.404001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.404118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.404232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.404347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.404461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.404575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.404697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.404814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.404928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.405042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.405156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.405280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.405423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.405542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.405657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.405772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.405886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.406001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.406123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.406239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.406377 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.406495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.406609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.406723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.406846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.406963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.407078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.407193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.407334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.407452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.407577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.407693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.407808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.407959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.408076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.408174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.408278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.408377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.408475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.408591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.408706 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.408821 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.408947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.409064 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.409179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.409300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.409300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.409546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.409674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.409792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.409906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.410021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.410135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.410257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.410381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.410498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.410612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.410727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.410842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.410957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.411080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.411196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.411311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.411425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.411540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.411654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.411778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.411894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.412008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.412123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.412237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.412352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.412474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.412590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.412705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.412820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.412935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.413049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.413171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.413287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.413416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.413533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.413647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.413763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.413889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.414006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.414122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.414237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.414351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.414473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.414590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.414704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.414859 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.414979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.415095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.415218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.415338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.415459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.415575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.415690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.415805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.415960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.416080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.416196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.416310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.416310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.416525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.416648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.416764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.416879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.417024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.417024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.417249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.417386 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.417511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.417511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.417726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.417726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.417940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.418108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.418229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.418344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.418459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.418459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.418673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.418796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.418913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.419028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.419143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.419257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.419371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.419494 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.419610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.419724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.419839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.419953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.420068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.420190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.420306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.420421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.420535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.420650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.420764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.420886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.421003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.421118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.421232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.421346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.421505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.421633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.421751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.421865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.421980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.422095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.422210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.422333 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.422450 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.422564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.422679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.422794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.422908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.423031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.423147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.423263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.423377 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.423999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424494 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424552 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] role.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] role.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.424983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.425971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.426086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.426201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.426316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.426440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.426556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.426671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.426786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.426900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427015 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427607 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.427947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.428162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.428276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.428391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.428514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.428631 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.428746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.428861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.428976 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.429091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.429217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.429335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.429468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.429585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.429699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.429814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.429941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.430058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.430174 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.430290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.430404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.430519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.430643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.430759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.430874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.430989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.431110 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.431234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.431351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.431466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.431581 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.431700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.431815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.431938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.432054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.432169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.432283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.432398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.432513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.432636 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.432752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.432867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.432981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.433096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.433217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.433341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.433480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.433597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.433714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.433829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.433943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.434070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.434186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.434301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.434415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.434530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.434644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.434767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.434884 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.434998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.435113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.435231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.435590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.435716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.435833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.435948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.436061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.436176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.436295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.436419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.436535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.436649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.436763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.436886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.437010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.437126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.437333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.437545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.437671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.437788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.437905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.438020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.438134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.438248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.438371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.438487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.438602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.438716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.438831 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.438946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.439069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.439773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.439901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.440003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.440100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.440197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.440319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.440435 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.440550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.440664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.440779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.440894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441248 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80288) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 13:13:00.441956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 12 13:13:00.441956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80287) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 12 13:13:00.471534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.471710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.471793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.471867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.471971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.472088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.480235 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.480478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.509443 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.509558 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.509636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.509723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.652904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.653429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.702598 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.702829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.703125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.703356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.767767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.767903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.768038 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.768222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.768320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.768435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.768519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.768646 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.768730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.768939 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.769153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.769364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.793294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.793638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.793748 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.793866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.793951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.794087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.794170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.794301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.794386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.794511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.794594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.794720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.794804 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.794928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.806103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.806421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.819674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.820000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.820226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.820311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.820451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.820534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.820663 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.820755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.820870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.820953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.821080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80288) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.821163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.821290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.821601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80287) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 12 13:13:00.838616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.838827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 13:13:00.839164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.839246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.839362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 13:13:00.839446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 13:13:00.839627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.839709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.839886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.839968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 12 13:13:00.840138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.840221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.840389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.840470 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.840639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.840890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.851791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.852064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.852154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.852330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.852416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.852610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.852696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.852925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.860803 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.861173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.864348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.864444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.864626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.864710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.864906 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.865009 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.865184 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.865267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.865484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.865569 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.865741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.865826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.865998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.866080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.866260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.866341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.866519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.866603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.866782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.866951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.867032 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.867234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.893962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.894603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.895560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.895835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.896203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.896478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.901022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.901298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.901707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.901790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.901982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.902259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.909497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.909775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.910226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.910504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.914204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.914488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.914763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.914930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.915025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.915215 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.915296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.915493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.915575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.915770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.915861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.916040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.916122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.916307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.916571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.916834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.937712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.937984 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.940201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.940472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.943068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.945070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.946683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.946976 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.948777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.949057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.952052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.954159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.956377 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.958497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.960024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.960320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.960609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.960892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.961163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.961457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.962116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.962410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.962698 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.962984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.963257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.963529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.980292 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.980576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.980838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.981098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.982694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.982976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.983240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.983498 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.988986 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.990703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.990978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.991370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.993051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.993321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.994228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.994504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.994777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.995051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.995325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.996556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:00.996835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.997113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.997421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:00.997805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.008951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.009225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.009522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.009810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.010085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.011608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.011699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.011885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.011967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.012161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.012444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.012718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.013005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.013272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.030254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.033160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.033769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.036309 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.037028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.038784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.039609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.040412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.040685 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.040963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.041200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.042976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.043251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.043534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.048278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.050041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.050976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.052576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.053212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.053521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.053912 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.054193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.055798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.056084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.056361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.056629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.063429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.063708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.063982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.066625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.066905 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.067179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.072338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80288) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.072615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.072886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.073152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.073434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.073800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.074080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.074353 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.074621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.074888 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.075096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80287) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 12 13:13:01.075178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.075370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.075461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.075644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.075726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.075905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80288) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.076035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.076166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.076432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.076706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.076975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.077334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.077651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.077931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.078191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.078462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.078725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80287) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 12 13:13:01.114273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55e3b73bc9c0 pid: 80288 (default app) Jul 12 13:13:01.116968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x55e3b73bc9c0 pid: 80287 (default app) Jul 12 13:13:01.117643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94821cdd-4d8b-449b-af98-08bd2281429e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:01.117937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94821cdd-4d8b-449b-af98-08bd2281429e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:01.118214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94821cdd-4d8b-449b-af98-08bd2281429e None None] PATH_INFO: `/v3` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:01.122410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Tue Jul 12 13:13:01 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:08.718421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb57ee7-9cfa-4c2e-b399-a76b569a31b1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:08.718951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb57ee7-9cfa-4c2e-b399-a76b569a31b1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:08.719046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb57ee7-9cfa-4c2e-b399-a76b569a31b1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:08.723297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:08.728673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:08.729122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:08.729238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:08.798068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: INFO dbcounter [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] Registered counter for database keystone Jul 12 13:13:08.820137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG oslo_db.sqlalchemy.engines [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80288) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 13:13:08.826296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writer thread running {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 13:13:09.006461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG passlib.handlers.bcrypt [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80288) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 13:13:09.006790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG passlib.handlers.bcrypt [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80288) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 13:13:09.061118 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:09.062765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:09.182308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 12 13:13:09.359995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb57ee7-9cfa-4c2e-b399-a76b569a31b1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:09.431457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: INFO dbcounter [None req-3eb57ee7-9cfa-4c2e-b399-a76b569a31b1 None None] Registered counter for database keystone Jul 12 13:13:09.451392 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG oslo_db.sqlalchemy.engines [None req-3eb57ee7-9cfa-4c2e-b399-a76b569a31b1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80287) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 12 13:13:09.457947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writer thread running {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 12 13:13:09.575901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb57ee7-9cfa-4c2e-b399-a76b569a31b1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:09.580576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a226bc0e-1d2e-43f7-ad5b-2ff083af8b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:09.581250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226bc0e-1d2e-43f7-ad5b-2ff083af8b11 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:09.581578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226bc0e-1d2e-43f7-ad5b-2ff083af8b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:09.581861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a226bc0e-1d2e-43f7-ad5b-2ff083af8b11 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:09.589890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226bc0e-1d2e-43f7-ad5b-2ff083af8b11 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:09.641557 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WARNING keystone.common.rbac_enforcer.enforcer [None req-a226bc0e-1d2e-43f7-ad5b-2ff083af8b11 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 13:13:09.641852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a226bc0e-1d2e-43f7-ad5b-2ff083af8b11 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:09.651584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Jul 12 13:13:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:13:09.657612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:09.672006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f559e-a6a2-4c66-b492-540c7a6cc789 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:09.677212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6662f8d-97be-466e-9e7f-107f677b6558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:09.678357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6662f8d-97be-466e-9e7f-107f677b6558 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:09.678649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6662f8d-97be-466e-9e7f-107f677b6558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:09.678930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6662f8d-97be-466e-9e7f-107f677b6558 admin admin] PATH_INFO: `/v3/projects/65457e2fc15c4d94825d2ac5ee397f82` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:09.683026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6662f8d-97be-466e-9e7f-107f677b6558 admin admin] RBAC: Authorizing `identity:get_project(project_id=65457e2fc15c4d94825d2ac5ee397f82)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:09.733414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b6662f8d-97be-466e-9e7f-107f677b6558 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 12 13:13:09.733955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6662f8d-97be-466e-9e7f-107f677b6558 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:09.738849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Tue Jul 12 13:13:09 2022] GET /identity/v3/projects/65457e2fc15c4d94825d2ac5ee397f82 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:11.716558 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b84eab-15c0-4989-b1f0-27c4602a512b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:11.717042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b84eab-15c0-4989-b1f0-27c4602a512b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:11.717138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b84eab-15c0-4989-b1f0-27c4602a512b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:11.721921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:11.727277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19b7a-e3c7-4e70-af97-4d2654c1087c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:11.727659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19b7a-e3c7-4e70-af97-4d2654c1087c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:11.727843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19b7a-e3c7-4e70-af97-4d2654c1087c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:11.747197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0fa19b7a-e3c7-4e70-af97-4d2654c1087c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:11.748724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0fa19b7a-e3c7-4e70-af97-4d2654c1087c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:11.757200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:11 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 12 13:13:11.778987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b84eab-15c0-4989-b1f0-27c4602a512b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:11.791397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b84eab-15c0-4989-b1f0-27c4602a512b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:11.795544 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0702a1-2998-4496-82d0-385eb7a49225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:11.796176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0702a1-2998-4496-82d0-385eb7a49225 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:11.796469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0702a1-2998-4496-82d0-385eb7a49225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:11.796748 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0702a1-2998-4496-82d0-385eb7a49225 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:11.805461 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0702a1-2998-4496-82d0-385eb7a49225 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:11.807252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0702a1-2998-4496-82d0-385eb7a49225 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:11.816842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Tue Jul 12 13:13:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:13:11.822117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa19b7a-e3c7-4e70-af97-4d2654c1087c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:11.832161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa19b7a-e3c7-4e70-af97-4d2654c1087c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:11.836508 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a86d56-e138-4880-ae52-0551dd8a5298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:11.837369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a86d56-e138-4880-ae52-0551dd8a5298 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:11.837690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a86d56-e138-4880-ae52-0551dd8a5298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:11.837976 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a86d56-e138-4880-ae52-0551dd8a5298 admin admin] PATH_INFO: `/v3/users/417a441bd00d4df89cd837724e786c1e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:11.839381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a86d56-e138-4880-ae52-0551dd8a5298 admin admin] RBAC: Authorizing `identity:get_user(user_id=417a441bd00d4df89cd837724e786c1e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:11.841213 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a86d56-e138-4880-ae52-0551dd8a5298 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:11.842817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Tue Jul 12 13:13:11 2022] GET /identity/v3/users/417a441bd00d4df89cd837724e786c1e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:13.909066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b1fce-758e-4639-971d-651f7ae97d04 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:13.909695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b1fce-758e-4639-971d-651f7ae97d04 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:13.909695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b1fce-758e-4639-971d-651f7ae97d04 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:13.911003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:13.916150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe86bd-76dc-4aa3-98b7-7d9cdb704321 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:13.916553 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe86bd-76dc-4aa3-98b7-7d9cdb704321 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:13.916716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fe86bd-76dc-4aa3-98b7-7d9cdb704321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:13.933681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f0fe86bd-76dc-4aa3-98b7-7d9cdb704321 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:13.934861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f0fe86bd-76dc-4aa3-98b7-7d9cdb704321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:13.982798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf170fa2-749c-41bf-bc73-0f0d7ca3edf4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:13.983089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf170fa2-749c-41bf-bc73-0f0d7ca3edf4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:13.983371 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf170fa2-749c-41bf-bc73-0f0d7ca3edf4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:13.984725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:13.990397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373c477-64e2-4700-b036-c239ed3934cc None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:13.990692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373c477-64e2-4700-b036-c239ed3934cc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:13.990974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e373c477-64e2-4700-b036-c239ed3934cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:14.017366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:14.030389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG passlib.handlers.bcrypt [None req-e373c477-64e2-4700-b036-c239ed3934cc None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80287) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 12 13:13:14.030820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG passlib.handlers.bcrypt [None req-e373c477-64e2-4700-b036-c239ed3934cc None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80287) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 12 13:13:14.040117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe86bd-76dc-4aa3-98b7-7d9cdb704321 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:14.065641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:14.077326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e373c477-64e2-4700-b036-c239ed3934cc None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:14.078963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e373c477-64e2-4700-b036-c239ed3934cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:14.146628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fe86bd-76dc-4aa3-98b7-7d9cdb704321 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:14.179730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:14.179902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d6eaa-1701-4ac8-b40d-922feed2ce6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:14.180457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d6eaa-1701-4ac8-b40d-922feed2ce6f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:14.180748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d6eaa-1701-4ac8-b40d-922feed2ce6f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:14.181030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933d6eaa-1701-4ac8-b40d-922feed2ce6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:14.193177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933d6eaa-1701-4ac8-b40d-922feed2ce6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:14.202125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373c477-64e2-4700-b036-c239ed3934cc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-933d6eaa-1701-4ac8-b40d-922feed2ce6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:14.204726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:14.205794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:14.206766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:13:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:13:14.211709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d6eaa-1701-4ac8-b40d-922feed2ce6f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:14.294532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:14.305420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e373c477-64e2-4700-b036-c239ed3934cc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:14.307693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:14.315433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933d6eaa-1701-4ac8-b40d-922feed2ce6f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:14.342260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7d7bc-089c-4e90-a1d3-b58f717f4d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:14.343153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7d7bc-089c-4e90-a1d3-b58f717f4d50 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:14.343440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7d7bc-089c-4e90-a1d3-b58f717f4d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:14.343717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e7d7bc-089c-4e90-a1d3-b58f717f4d50 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:14.350008 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07bf04-1c70-458f-89ca-4406946606cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:14.350682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07bf04-1c70-458f-89ca-4406946606cf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:14.350971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07bf04-1c70-458f-89ca-4406946606cf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:14.351270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de07bf04-1c70-458f-89ca-4406946606cf None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:14.352042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07bf04-1c70-458f-89ca-4406946606cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:14.352354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07bf04-1c70-458f-89ca-4406946606cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:14.353781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de07bf04-1c70-458f-89ca-4406946606cf None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:14.366210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:14.371686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07bf04-1c70-458f-89ca-4406946606cf None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:14.412308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7d7bc-089c-4e90-a1d3-b58f717f4d50 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:14.413855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e7d7bc-089c-4e90-a1d3-b58f717f4d50 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:14.470779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de07bf04-1c70-458f-89ca-4406946606cf None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:14.506199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707949a-1291-47ff-9a4a-6fa940b1380c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:14.507146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707949a-1291-47ff-9a4a-6fa940b1380c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:14.507444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707949a-1291-47ff-9a4a-6fa940b1380c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:14.507720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f707949a-1291-47ff-9a4a-6fa940b1380c None admin] PATH_INFO: `/v3/users/417a441bd00d4df89cd837724e786c1e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:14.508986 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707949a-1291-47ff-9a4a-6fa940b1380c None admin] RBAC: Authorizing `identity:get_user(user_id=417a441bd00d4df89cd837724e786c1e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:14.510460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f707949a-1291-47ff-9a4a-6fa940b1380c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:14.512085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 9/16] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:14 2022] GET /identity/v3/users/417a441bd00d4df89cd837724e786c1e => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:14.519488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 8/17] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:13:14.519617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707949a-1291-47ff-9a4a-6fa940b1380c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:14.520485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7d7bc-089c-4e90-a1d3-b58f717f4d50 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:14.523302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:14.532973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:14.617911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f707949a-1291-47ff-9a4a-6fa940b1380c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:14.618503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e7d7bc-089c-4e90-a1d3-b58f717f4d50 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:14.651656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e222c41-62ac-4780-855f-b3a03ad9d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:14.652311 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ab071-2327-45f7-aad3-bf824208363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:14.652453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e222c41-62ac-4780-855f-b3a03ad9d96c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:14.652742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e222c41-62ac-4780-855f-b3a03ad9d96c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:14.653023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e222c41-62ac-4780-855f-b3a03ad9d96c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:14.653108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ab071-2327-45f7-aad3-bf824208363f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:14.653391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ab071-2327-45f7-aad3-bf824208363f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:14.653677 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91ab071-2327-45f7-aad3-bf824208363f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:14.654859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ab071-2327-45f7-aad3-bf824208363f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:14.656310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91ab071-2327-45f7-aad3-bf824208363f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:14.658002 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 10/18] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:14.663858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ab071-2327-45f7-aad3-bf824208363f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:14.671729 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e222c41-62ac-4780-855f-b3a03ad9d96c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:14.673820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e222c41-62ac-4780-855f-b3a03ad9d96c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0e222c41-62ac-4780-855f-b3a03ad9d96c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:13:14.697217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:14.698250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:14.699062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:13:14.699062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:13:14.699062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:13:14.699062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:14.699062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 9/19] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:13:14.705863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e222c41-62ac-4780-855f-b3a03ad9d96c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:14.743810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:14.762422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:14.763858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91ab071-2327-45f7-aad3-bf824208363f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:14.796772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a780ce0-35c6-42ff-be4e-57ce2b02e3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:14.797448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a780ce0-35c6-42ff-be4e-57ce2b02e3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:14.797735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a780ce0-35c6-42ff-be4e-57ce2b02e3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:14.798012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a780ce0-35c6-42ff-be4e-57ce2b02e3e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:14.799557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a780ce0-35c6-42ff-be4e-57ce2b02e3e9 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.domain.id=default, user.id=417a441bd00d4df89cd837724e786c1e) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:14.799877 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a780ce0-35c6-42ff-be4e-57ce2b02e3e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:14.801310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a780ce0-35c6-42ff-be4e-57ce2b02e3e9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:14.806399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e222c41-62ac-4780-855f-b3a03ad9d96c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:14.807824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 11/20] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Jul 12 13:13:14 2022] GET /identity/v3/role_assignments?user.id=417a441bd00d4df89cd837724e786c1e&scope.domain.id=default&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:14.842687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f07802-1939-471f-a6ef-9d7159b2ad91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:14.843372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f07802-1939-471f-a6ef-9d7159b2ad91 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:14.843663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f07802-1939-471f-a6ef-9d7159b2ad91 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:14.843944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f07802-1939-471f-a6ef-9d7159b2ad91 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:14.844697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f07802-1939-471f-a6ef-9d7159b2ad91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:14.845091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f07802-1939-471f-a6ef-9d7159b2ad91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:14.846574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f07802-1939-471f-a6ef-9d7159b2ad91 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:14.866727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 10/21] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:13:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:17.203765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b9ae2-7ec7-4db0-a0c0-b7a68f4497a7 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:17.204794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b9ae2-7ec7-4db0-a0c0-b7a68f4497a7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:17.205047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b9ae2-7ec7-4db0-a0c0-b7a68f4497a7 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:17.206447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 12/22] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:17.211900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48958f7-ddac-45a2-bf18-55076b69dbda None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:17.212722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48958f7-ddac-45a2-bf18-55076b69dbda None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:17.213207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48958f7-ddac-45a2-bf18-55076b69dbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:17.243117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b48958f7-ddac-45a2-bf18-55076b69dbda None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:17.244696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b48958f7-ddac-45a2-bf18-55076b69dbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:17.335587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 11/23] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:17.369946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b9ae2-7ec7-4db0-a0c0-b7a68f4497a7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:17.388407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50dea4-e308-44ae-95b8-a67f3a3bbf13 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:17.389032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50dea4-e308-44ae-95b8-a67f3a3bbf13 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:17.389637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a50dea4-e308-44ae-95b8-a67f3a3bbf13 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:17.391353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:17.405799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907454f0-5331-4cdb-9e9c-41c067e1d811 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:17.406396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907454f0-5331-4cdb-9e9c-41c067e1d811 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:17.406899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907454f0-5331-4cdb-9e9c-41c067e1d811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:17.438492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-907454f0-5331-4cdb-9e9c-41c067e1d811 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:17.440849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-907454f0-5331-4cdb-9e9c-41c067e1d811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:17.495627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:17.547852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:17.550075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3ffbb-fbe0-42ca-adc7-8075119d3fde None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:17.550530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3ffbb-fbe0-42ca-adc7-8075119d3fde None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:17.550944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d3ffbb-fbe0-42ca-adc7-8075119d3fde None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:17.552359 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 14/26] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:17.558033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1558bd-ec3c-4c17-891a-57537de4d819 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:17.558606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1558bd-ec3c-4c17-891a-57537de4d819 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:17.559053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1558bd-ec3c-4c17-891a-57537de4d819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:17.559930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b9ae2-7ec7-4db0-a0c0-b7a68f4497a7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:17.601422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2d1558bd-ec3c-4c17-891a-57537de4d819 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:17.602976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2d1558bd-ec3c-4c17-891a-57537de4d819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:17.616849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46d472-64ae-4b5b-ab3d-196a3ebdbc1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:17.618047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46d472-64ae-4b5b-ab3d-196a3ebdbc1c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:17.618531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46d472-64ae-4b5b-ab3d-196a3ebdbc1c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:17.619003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46d472-64ae-4b5b-ab3d-196a3ebdbc1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:17.643012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46d472-64ae-4b5b-ab3d-196a3ebdbc1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9e46d472-64ae-4b5b-ab3d-196a3ebdbc1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:17.651334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:17.652225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:17.652968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:17.655374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 13/27] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:13:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:17.656374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46d472-64ae-4b5b-ab3d-196a3ebdbc1c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:17.663640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:17.750219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 15/28] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:17.751570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1558bd-ec3c-4c17-891a-57537de4d819 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:17.809056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46d472-64ae-4b5b-ab3d-196a3ebdbc1c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:17.844781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed9bf1-461c-40db-bb1e-3dfdd5c38d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:17.845988 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed9bf1-461c-40db-bb1e-3dfdd5c38d98 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:17.846474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed9bf1-461c-40db-bb1e-3dfdd5c38d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:17.846969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed9bf1-461c-40db-bb1e-3dfdd5c38d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:17.857824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed9bf1-461c-40db-bb1e-3dfdd5c38d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:17.859624 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed9bf1-461c-40db-bb1e-3dfdd5c38d98 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-76ed9bf1-461c-40db-bb1e-3dfdd5c38d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:13:17.879946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:17.880770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:17.881577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:13:17.881577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:13:17.881577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:13:17.881577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:17.883041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 14/29] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:17.883925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed9bf1-461c-40db-bb1e-3dfdd5c38d98 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:17.897780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1558bd-ec3c-4c17-891a-57537de4d819 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:17.921735 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:17.952312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:17.994268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a119de0-34fd-4854-8337-f66f2063f50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:17.994792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a119de0-34fd-4854-8337-f66f2063f50f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:17.995089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a119de0-34fd-4854-8337-f66f2063f50f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:17.995395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a119de0-34fd-4854-8337-f66f2063f50f None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:17.996143 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a119de0-34fd-4854-8337-f66f2063f50f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:17.996455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a119de0-34fd-4854-8337-f66f2063f50f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:17.997894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a119de0-34fd-4854-8337-f66f2063f50f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:18.001814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed9bf1-461c-40db-bb1e-3dfdd5c38d98 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:18.028762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 16/30] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:18.029878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a119de0-34fd-4854-8337-f66f2063f50f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:18.037271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30e327-4506-460e-b498-dd407ec38a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:18.038175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30e327-4506-460e-b498-dd407ec38a83 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.038604 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30e327-4506-460e-b498-dd407ec38a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.039019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd30e327-4506-460e-b498-dd407ec38a83 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.039939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30e327-4506-460e-b498-dd407ec38a83 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:18.041515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd30e327-4506-460e-b498-dd407ec38a83 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:18.159472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a119de0-34fd-4854-8337-f66f2063f50f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:18.198025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcb8ced-002a-4d56-93cc-435775750dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:18.198997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb8ced-002a-4d56-93cc-435775750dcf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.199458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb8ced-002a-4d56-93cc-435775750dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.199885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcb8ced-002a-4d56-93cc-435775750dcf None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.200793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb8ced-002a-4d56-93cc-435775750dcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:18.201258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb8ced-002a-4d56-93cc-435775750dcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:18.202805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcb8ced-002a-4d56-93cc-435775750dcf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:18.217424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 15/31] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 12 13:13:17 2022] POST /identity/v3/roles => generated 219 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:18.217424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30e327-4506-460e-b498-dd407ec38a83 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:18.228996 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 17/32] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:13:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:18.230996 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301a8bd-0950-440f-966e-a87cc68a63ce None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.231422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301a8bd-0950-440f-966e-a87cc68a63ce None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.231855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c301a8bd-0950-440f-966e-a87cc68a63ce None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.233243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 18/33] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:18.241816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301a8bd-0950-440f-966e-a87cc68a63ce None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:18.306387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:18.344817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd30e327-4506-460e-b498-dd407ec38a83 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:18.365944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:18.389178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b355c5b-5eac-408e-8dd9-e3dcdbd0ec24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:18.390807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b355c5b-5eac-408e-8dd9-e3dcdbd0ec24 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.391740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b355c5b-5eac-408e-8dd9-e3dcdbd0ec24 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.392319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b355c5b-5eac-408e-8dd9-e3dcdbd0ec24 None admin] PATH_INFO: `/v3/users/417a441bd00d4df89cd837724e786c1e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.394616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b355c5b-5eac-408e-8dd9-e3dcdbd0ec24 None admin] RBAC: Authorizing `identity:get_user(user_id=417a441bd00d4df89cd837724e786c1e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:18.396693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b355c5b-5eac-408e-8dd9-e3dcdbd0ec24 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:18.398948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 16/34] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:18 2022] GET /identity/v3/users/417a441bd00d4df89cd837724e786c1e => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:18.400464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332fd9b-7260-45f6-82e7-d174da51cca6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.400989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332fd9b-7260-45f6-82e7-d174da51cca6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.401515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2332fd9b-7260-45f6-82e7-d174da51cca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.423976 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-2332fd9b-7260-45f6-82e7-d174da51cca6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:18.426692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2332fd9b-7260-45f6-82e7-d174da51cca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:18.483672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c301a8bd-0950-440f-966e-a87cc68a63ce None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:18.504646 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 17/35] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:18.505855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332fd9b-7260-45f6-82e7-d174da51cca6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:18.524785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79118725-8093-4d25-813a-1ba8a25472c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:18.525653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79118725-8093-4d25-813a-1ba8a25472c1 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.525946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79118725-8093-4d25-813a-1ba8a25472c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.526231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79118725-8093-4d25-813a-1ba8a25472c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.527375 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79118725-8093-4d25-813a-1ba8a25472c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:18.529010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79118725-8093-4d25-813a-1ba8a25472c1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:18.618998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2332fd9b-7260-45f6-82e7-d174da51cca6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:18.634154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:18.656228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713a3a8-8e42-4409-a311-2a488d7590bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:18.657227 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713a3a8-8e42-4409-a311-2a488d7590bc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.658439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713a3a8-8e42-4409-a311-2a488d7590bc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.658826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9713a3a8-8e42-4409-a311-2a488d7590bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.660144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713a3a8-8e42-4409-a311-2a488d7590bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:18.661729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9713a3a8-8e42-4409-a311-2a488d7590bc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:18.667961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 18/36] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:18.667961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713a3a8-8e42-4409-a311-2a488d7590bc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:18.697982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 19/37] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:13:18 2022] POST /identity/v3/projects => generated 301 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:18.699474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79118725-8093-4d25-813a-1ba8a25472c1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:18.808004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9713a3a8-8e42-4409-a311-2a488d7590bc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:18.816115 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:18.838295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79118725-8093-4d25-813a-1ba8a25472c1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:18.852220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4436589-5b89-4809-a5d4-21a7c593cb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:18.852910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4436589-5b89-4809-a5d4-21a7c593cb22 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.853201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4436589-5b89-4809-a5d4-21a7c593cb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.853508 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4436589-5b89-4809-a5d4-21a7c593cb22 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.854279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4436589-5b89-4809-a5d4-21a7c593cb22 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:18.855720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4436589-5b89-4809-a5d4-21a7c593cb22 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:18.872665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 19/38] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 12 13:13:18 2022] POST /identity/v3/roles => generated 225 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:18.873956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac40a1-8f3b-4431-b8f0-32c4f95ac75d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.874248 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac40a1-8f3b-4431-b8f0-32c4f95ac75d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.874527 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ac40a1-8f3b-4431-b8f0-32c4f95ac75d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.876169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 20/39] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:18.877151 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867e369-bef0-4142-8987-ee7642c7057b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.877470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867e369-bef0-4142-8987-ee7642c7057b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.877783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a867e369-bef0-4142-8987-ee7642c7057b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.879186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 21/40] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:18.884454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4cb76-1bfd-4797-aea4-62cbf5f77eae None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.884711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4cb76-1bfd-4797-aea4-62cbf5f77eae None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.884998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f4cb76-1bfd-4797-aea4-62cbf5f77eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.892278 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7688320-cb6d-489c-8e2d-85e17ee5e3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:18.892278 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7688320-cb6d-489c-8e2d-85e17ee5e3f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.892616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7688320-cb6d-489c-8e2d-85e17ee5e3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.892861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7688320-cb6d-489c-8e2d-85e17ee5e3f2 None admin] PATH_INFO: `/v3/domains/default/users/417a441bd00d4df89cd837724e786c1e/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.900594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7688320-cb6d-489c-8e2d-85e17ee5e3f2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=417a441bd00d4df89cd837724e786c1e, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:18.902680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7688320-cb6d-489c-8e2d-85e17ee5e3f2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:18.907115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b9f4cb76-1bfd-4797-aea4-62cbf5f77eae None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:18.908985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b9f4cb76-1bfd-4797-aea4-62cbf5f77eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:18.932625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 20/41] 10.222.0.23 () {58 vars in 1361 bytes} [Tue Jul 12 13:13:18 2022] PUT /identity/v3/domains/default/users/417a441bd00d4df89cd837724e786c1e/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:13:18.934763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fe6e0-5316-419e-a0bc-e331e6188dce None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:18.934763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fe6e0-5316-419e-a0bc-e331e6188dce None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:18.934763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132fe6e0-5316-419e-a0bc-e331e6188dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:18.960382 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:18.962817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-132fe6e0-5316-419e-a0bc-e331e6188dce None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:18.964637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-132fe6e0-5316-419e-a0bc-e331e6188dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:19.008618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 22/42] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:19.040495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4cb76-1bfd-4797-aea4-62cbf5f77eae None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:19.084831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 21/43] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:19.111131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fe6e0-5316-419e-a0bc-e331e6188dce None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:19.197436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:19.238229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132fe6e0-5316-419e-a0bc-e331e6188dce None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:19.276771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfdf47-a905-45c7-a562-d8c6a321056b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:19.277478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfdf47-a905-45c7-a562-d8c6a321056b None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:19.277775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfdf47-a905-45c7-a562-d8c6a321056b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:19.278062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfdf47-a905-45c7-a562-d8c6a321056b None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:19.278824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfdf47-a905-45c7-a562-d8c6a321056b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:19.280231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfdf47-a905-45c7-a562-d8c6a321056b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:19.280623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f4cb76-1bfd-4797-aea4-62cbf5f77eae None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:19.294838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 22/44] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 12 13:13:19 2022] POST /identity/v3/roles => generated 223 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:13:19.322474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5f8a91-dec1-4a01-9dc5-c032a50ddf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:19.323472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f8a91-dec1-4a01-9dc5-c032a50ddf6a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:19.323867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f8a91-dec1-4a01-9dc5-c032a50ddf6a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:19.324266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5f8a91-dec1-4a01-9dc5-c032a50ddf6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:19.325731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f8a91-dec1-4a01-9dc5-c032a50ddf6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:19.327327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5f8a91-dec1-4a01-9dc5-c032a50ddf6a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:19.334132 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 23/45] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:19.335746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfdf47-a905-45c7-a562-d8c6a321056b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:19.460544 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfdf47-a905-45c7-a562-d8c6a321056b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:19.490480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:19.497112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae15e2c-8ce9-4bf9-ba4b-e4a1f0381540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:19.498178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae15e2c-8ce9-4bf9-ba4b-e4a1f0381540 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:19.498178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae15e2c-8ce9-4bf9-ba4b-e4a1f0381540 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:19.498360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae15e2c-8ce9-4bf9-ba4b-e4a1f0381540 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:19.499097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae15e2c-8ce9-4bf9-ba4b-e4a1f0381540 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:19.502259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae15e2c-8ce9-4bf9-ba4b-e4a1f0381540 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:19.530472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 23/46] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:13:19 2022] POST /identity/v3/projects => generated 312 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:21.017464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceffa74-7c6e-423c-adcd-43d94a845e22 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.018019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceffa74-7c6e-423c-adcd-43d94a845e22 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.018019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceffa74-7c6e-423c-adcd-43d94a845e22 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.019533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 24/47] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:21.024406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7932ada-9db5-491e-9762-9aed837e8c1f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.024784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7932ada-9db5-491e-9762-9aed837e8c1f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.024972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7932ada-9db5-491e-9762-9aed837e8c1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.041682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b7932ada-9db5-491e-9762-9aed837e8c1f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:21.042858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b7932ada-9db5-491e-9762-9aed837e8c1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:21.108084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 24/48] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:21.131200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceffa74-7c6e-423c-adcd-43d94a845e22 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:21.174192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:21.234618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceffa74-7c6e-423c-adcd-43d94a845e22 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:21.273966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc72b2-2f82-4320-ba82-f3fa17d26673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:21.274717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc72b2-2f82-4320-ba82-f3fa17d26673 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.275007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc72b2-2f82-4320-ba82-f3fa17d26673 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.275291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc72b2-2f82-4320-ba82-f3fa17d26673 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.286063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc72b2-2f82-4320-ba82-f3fa17d26673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-6bcc72b2-2f82-4320-ba82-f3fa17d26673 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:21.293855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:21.294723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:21.295523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 25/49] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:13:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:21.301367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7932ada-9db5-491e-9762-9aed837e8c1f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:21.397798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7932ada-9db5-491e-9762-9aed837e8c1f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:21.410830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:21.432436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329d2ab-9ce9-473c-b31e-4920d2829fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:21.433101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329d2ab-9ce9-473c-b31e-4920d2829fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.433417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329d2ab-9ce9-473c-b31e-4920d2829fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.433703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329d2ab-9ce9-473c-b31e-4920d2829fc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.434458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329d2ab-9ce9-473c-b31e-4920d2829fc0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:21.434768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329d2ab-9ce9-473c-b31e-4920d2829fc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:21.436426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329d2ab-9ce9-473c-b31e-4920d2829fc0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:21.445297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 25/50] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:21.450732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc72b2-2f82-4320-ba82-f3fa17d26673 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:21.545777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:21.549249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc72b2-2f82-4320-ba82-f3fa17d26673 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:21.583539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec94f88-e4b6-4a56-8f57-753a01be5e2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:21.584424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec94f88-e4b6-4a56-8f57-753a01be5e2e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.584708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec94f88-e4b6-4a56-8f57-753a01be5e2e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.584985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec94f88-e4b6-4a56-8f57-753a01be5e2e None admin] PATH_INFO: `/v3/users/417a441bd00d4df89cd837724e786c1e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.586217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec94f88-e4b6-4a56-8f57-753a01be5e2e None admin] RBAC: Authorizing `identity:get_user(user_id=417a441bd00d4df89cd837724e786c1e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:21.587744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec94f88-e4b6-4a56-8f57-753a01be5e2e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:21.589403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 26/51] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:21 2022] GET /identity/v3/users/417a441bd00d4df89cd837724e786c1e => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:21.594506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff31dd-7440-4c2f-a300-7de0e3839303 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.594644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec94f88-e4b6-4a56-8f57-753a01be5e2e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:21.594835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff31dd-7440-4c2f-a300-7de0e3839303 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.595170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff31dd-7440-4c2f-a300-7de0e3839303 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.596546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 26/52] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:21.601489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2d899-1206-4222-acdf-661b2b00f9bb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.601779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2d899-1206-4222-acdf-661b2b00f9bb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.602057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2d899-1206-4222-acdf-661b2b00f9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.619134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e3e2d899-1206-4222-acdf-661b2b00f9bb None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:21.620340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e3e2d899-1206-4222-acdf-661b2b00f9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:21.696590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec94f88-e4b6-4a56-8f57-753a01be5e2e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:21.731412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301cca5-4d20-4946-b507-08cffe16e384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:21.732228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301cca5-4d20-4946-b507-08cffe16e384 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.732514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301cca5-4d20-4946-b507-08cffe16e384 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.732790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a301cca5-4d20-4946-b507-08cffe16e384 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.734037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301cca5-4d20-4946-b507-08cffe16e384 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:21.736188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a301cca5-4d20-4946-b507-08cffe16e384 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:21.738330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 27/53] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:21.744103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301cca5-4d20-4946-b507-08cffe16e384 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:21.780312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:21.792901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 27/54] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:21.816418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2d899-1206-4222-acdf-661b2b00f9bb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:21.848308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a301cca5-4d20-4946-b507-08cffe16e384 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:21.882131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb42b2-f6ec-4dab-b7b3-c4cb2892e621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:21.882837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb42b2-f6ec-4dab-b7b3-c4cb2892e621 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.883152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb42b2-f6ec-4dab-b7b3-c4cb2892e621 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.883509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeb42b2-f6ec-4dab-b7b3-c4cb2892e621 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.884603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb42b2-f6ec-4dab-b7b3-c4cb2892e621 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.domain.id=default, user.id=417a441bd00d4df89cd837724e786c1e) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:21.884873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb42b2-f6ec-4dab-b7b3-c4cb2892e621 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:21.886469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeb42b2-f6ec-4dab-b7b3-c4cb2892e621 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:21.893112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 28/55] 10.222.0.23 () {56 vars in 1322 bytes} [Tue Jul 12 13:13:21 2022] GET /identity/v3/role_assignments?user.id=417a441bd00d4df89cd837724e786c1e&scope.domain.id=default&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:21.927431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2d899-1206-4222-acdf-661b2b00f9bb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:21.930191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:21.965370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4393c1-fa8b-49e0-b1c7-954ce2b4b7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:21.966208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4393c1-fa8b-49e0-b1c7-954ce2b4b7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:21.966496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4393c1-fa8b-49e0-b1c7-954ce2b4b7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:21.966778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4393c1-fa8b-49e0-b1c7-954ce2b4b7e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:21.967994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4393c1-fa8b-49e0-b1c7-954ce2b4b7e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:21.969521 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4393c1-fa8b-49e0-b1c7-954ce2b4b7e4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:21.971950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 28/56] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:21.977780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb42b2-f6ec-4dab-b7b3-c4cb2892e621 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:22.079923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:22.082844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeb42b2-f6ec-4dab-b7b3-c4cb2892e621 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:22.117559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7ac4c-4c5c-4f99-bc83-0ac22fb89d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:22.118229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7ac4c-4c5c-4f99-bc83-0ac22fb89d12 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:22.118518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7ac4c-4c5c-4f99-bc83-0ac22fb89d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:22.118799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7ac4c-4c5c-4f99-bc83-0ac22fb89d12 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:22.119530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7ac4c-4c5c-4f99-bc83-0ac22fb89d12 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:22.120975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7ac4c-4c5c-4f99-bc83-0ac22fb89d12 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:22.147718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 29/57] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Jul 12 13:13:21 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:24.184330 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53315fc8-94be-4fb0-8add-570f35717076 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:24.184874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53315fc8-94be-4fb0-8add-570f35717076 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:24.184874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53315fc8-94be-4fb0-8add-570f35717076 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:24.186552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 29/58] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:24.192231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0343bb-e3a6-4998-92b0-0d42759d42d1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:24.192601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0343bb-e3a6-4998-92b0-0d42759d42d1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:24.192791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0343bb-e3a6-4998-92b0-0d42759d42d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:24.209892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fc0343bb-e3a6-4998-92b0-0d42759d42d1 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:24.211105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fc0343bb-e3a6-4998-92b0-0d42759d42d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:24.277136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 30/59] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:24.301109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53315fc8-94be-4fb0-8add-570f35717076 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:24.399161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53315fc8-94be-4fb0-8add-570f35717076 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:24.431479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edb443-bcf8-427c-83db-bedace15c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:24.432255 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb443-bcf8-427c-83db-bedace15c5cd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:24.432540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb443-bcf8-427c-83db-bedace15c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:24.432812 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edb443-bcf8-427c-83db-bedace15c5cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:24.433995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edb443-bcf8-427c-83db-bedace15c5cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:24.435738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edb443-bcf8-427c-83db-bedace15c5cd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:24.437254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 30/60] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:24.443242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0343bb-e3a6-4998-92b0-0d42759d42d1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:24.521786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:24.541037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0343bb-e3a6-4998-92b0-0d42759d42d1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:24.574781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e51c3b-d2c3-4c97-9c6e-7a506733f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:24.575454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e51c3b-d2c3-4c97-9c6e-7a506733f4c5 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:24.575745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e51c3b-d2c3-4c97-9c6e-7a506733f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:24.576027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e51c3b-d2c3-4c97-9c6e-7a506733f4c5 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:24.576745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e51c3b-d2c3-4c97-9c6e-7a506733f4c5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:24.578221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e51c3b-d2c3-4c97-9c6e-7a506733f4c5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:24.602272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 31/61] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:13:24 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:27.564562 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae01587-faab-4db6-8ca5-08b815392bc9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:27.565062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae01587-faab-4db6-8ca5-08b815392bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:27.565062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae01587-faab-4db6-8ca5-08b815392bc9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:27.566461 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 31/62] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:27.571796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208dbe-7ee7-4cd1-be9a-f23c51c01d9a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:27.572642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208dbe-7ee7-4cd1-be9a-f23c51c01d9a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:27.573146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc208dbe-7ee7-4cd1-be9a-f23c51c01d9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:27.599308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fc208dbe-7ee7-4cd1-be9a-f23c51c01d9a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:27.600830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fc208dbe-7ee7-4cd1-be9a-f23c51c01d9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:27.683673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 32/63] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:27.734842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae01587-faab-4db6-8ca5-08b815392bc9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:27.790391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:27.874131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae01587-faab-4db6-8ca5-08b815392bc9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:27.922282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f8a5a3-1f73-443d-b8cd-46536b8de99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:27.923743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8a5a3-1f73-443d-b8cd-46536b8de99d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:27.923743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8a5a3-1f73-443d-b8cd-46536b8de99d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:27.924485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f8a5a3-1f73-443d-b8cd-46536b8de99d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:27.944397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f8a5a3-1f73-443d-b8cd-46536b8de99d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-69f8a5a3-1f73-443d-b8cd-46536b8de99d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:27.953719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:27.954897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:27.955808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 32/64] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:27.961445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208dbe-7ee7-4cd1-be9a-f23c51c01d9a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:28.051160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:28.059187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267832e5-21ab-46f9-84cf-5d12f104f644 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.059890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267832e5-21ab-46f9-84cf-5d12f104f644 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.060628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267832e5-21ab-46f9-84cf-5d12f104f644 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.062307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 33/65] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:28.072608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002bb55-9af1-4235-9bcb-26410697f699 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.073503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002bb55-9af1-4235-9bcb-26410697f699 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.074024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002bb55-9af1-4235-9bcb-26410697f699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.105784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e002bb55-9af1-4235-9bcb-26410697f699 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:28.108455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e002bb55-9af1-4235-9bcb-26410697f699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:28.114584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc208dbe-7ee7-4cd1-be9a-f23c51c01d9a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:28.154292 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23525d45-5b32-4025-92de-403c1ba7ef4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:28.155435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525d45-5b32-4025-92de-403c1ba7ef4f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.156774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525d45-5b32-4025-92de-403c1ba7ef4f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.160121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23525d45-5b32-4025-92de-403c1ba7ef4f None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.161113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525d45-5b32-4025-92de-403c1ba7ef4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:28.161664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525d45-5b32-4025-92de-403c1ba7ef4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:28.163292 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23525d45-5b32-4025-92de-403c1ba7ef4f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:28.178640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 33/66] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:28.181534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3628aa-e503-45be-a287-87254ea0a5d6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.182075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3628aa-e503-45be-a287-87254ea0a5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.182560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3628aa-e503-45be-a287-87254ea0a5d6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.184187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 34/67] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:28.189731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3628aa-e503-45be-a287-87254ea0a5d6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:28.233516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 34/68] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:28.235806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b564dcf-0468-4b77-8ecf-a8b7df661185 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.236590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b564dcf-0468-4b77-8ecf-a8b7df661185 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.236590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b564dcf-0468-4b77-8ecf-a8b7df661185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.260437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6b564dcf-0468-4b77-8ecf-a8b7df661185 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:28.261670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6b564dcf-0468-4b77-8ecf-a8b7df661185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:28.286967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:28.325958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3628aa-e503-45be-a287-87254ea0a5d6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:28.341396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 35/69] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:28.341396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c13c2-a789-4fb1-b963-748d4d5f664e None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.341396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c13c2-a789-4fb1-b963-748d4d5f664e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.341396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868c13c2-a789-4fb1-b963-748d4d5f664e None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.341396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 36/70] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:28.342463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c13c2-a789-4fb1-b963-748d4d5f664e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:28.364044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:28.372302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a4f571-9406-4c86-9667-c35f6e89d3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:28.373188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4f571-9406-4c86-9667-c35f6e89d3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.373514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4f571-9406-4c86-9667-c35f6e89d3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.373795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a4f571-9406-4c86-9667-c35f6e89d3d6 None admin] PATH_INFO: `/v3/users/4bd8fa54875c432abf00243484dacb64` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.388653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4f571-9406-4c86-9667-c35f6e89d3d6 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd8fa54875c432abf00243484dacb64)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:28.390342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a4f571-9406-4c86-9667-c35f6e89d3d6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:28.396423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 35/71] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:28 2022] GET /identity/v3/users/4bd8fa54875c432abf00243484dacb64 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:28.396423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1af664-61a4-4beb-bfdd-f0de1c619977 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.396423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1af664-61a4-4beb-bfdd-f0de1c619977 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.396423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1af664-61a4-4beb-bfdd-f0de1c619977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.411960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3a1af664-61a4-4beb-bfdd-f0de1c619977 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:28.413092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3a1af664-61a4-4beb-bfdd-f0de1c619977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:28.452455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868c13c2-a789-4fb1-b963-748d4d5f664e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:28.478875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 36/72] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:28.479704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1af664-61a4-4beb-bfdd-f0de1c619977 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:28.486332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e608-1110-4504-8c98-3921fca8c314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:28.487141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e608-1110-4504-8c98-3921fca8c314 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.487425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e608-1110-4504-8c98-3921fca8c314 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.487700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2802e608-1110-4504-8c98-3921fca8c314 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.497625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2802e608-1110-4504-8c98-3921fca8c314 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2802e608-1110-4504-8c98-3921fca8c314 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:28.506623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:28.507721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 37/73] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:13:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:28.508673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e608-1110-4504-8c98-3921fca8c314 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:28.530753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:28.601069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1af664-61a4-4beb-bfdd-f0de1c619977 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:28.611907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2802e608-1110-4504-8c98-3921fca8c314 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:28.618648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:28.635340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717bc09d-913e-4981-a42a-6fc34d3256d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:28.636161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bc09d-913e-4981-a42a-6fc34d3256d1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.636439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bc09d-913e-4981-a42a-6fc34d3256d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.636718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717bc09d-913e-4981-a42a-6fc34d3256d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.645716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775aed7-e582-46fd-87f5-ed66540f4ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:28.646373 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775aed7-e582-46fd-87f5-ed66540f4ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.646664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775aed7-e582-46fd-87f5-ed66540f4ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.646788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717bc09d-913e-4981-a42a-6fc34d3256d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:28.646951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4775aed7-e582-46fd-87f5-ed66540f4ca0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-717bc09d-913e-4981-a42a-6fc34d3256d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:28.654440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:28.655301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 37/74] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:13:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:28.661611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a621f-9a4a-40a2-a405-988d630bc8c2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.662322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a621f-9a4a-40a2-a405-988d630bc8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.662322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737a621f-9a4a-40a2-a405-988d630bc8c2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.662322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 38/75] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:28.662322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a621f-9a4a-40a2-a405-988d630bc8c2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:28.721130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775aed7-e582-46fd-87f5-ed66540f4ca0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:28.722675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4775aed7-e582-46fd-87f5-ed66540f4ca0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:28.743019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:28.760578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737a621f-9a4a-40a2-a405-988d630bc8c2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:28.793200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617bc4-1f76-4962-958a-961601a404cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:28.794030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617bc4-1f76-4962-958a-961601a404cb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.794318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617bc4-1f76-4962-958a-961601a404cb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.794600 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f617bc4-1f76-4962-958a-961601a404cb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.805358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f617bc4-1f76-4962-958a-961601a404cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-5f617bc4-1f76-4962-958a-961601a404cb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:28.813365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:28.814448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:28.815290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 39/76] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:28.815907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617bc4-1f76-4962-958a-961601a404cb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:28.824507 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 38/77] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:28.825346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775aed7-e582-46fd-87f5-ed66540f4ca0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:28.854799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:28.916134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f617bc4-1f76-4962-958a-961601a404cb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:28.924861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4775aed7-e582-46fd-87f5-ed66540f4ca0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:28.945709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:28.950557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7883062c-96ba-461b-9b90-567aa23dba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:28.951229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883062c-96ba-461b-9b90-567aa23dba18 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.951520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883062c-96ba-461b-9b90-567aa23dba18 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.951803 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7883062c-96ba-461b-9b90-567aa23dba18 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.952559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883062c-96ba-461b-9b90-567aa23dba18 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:28.952871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883062c-96ba-461b-9b90-567aa23dba18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:28.954342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7883062c-96ba-461b-9b90-567aa23dba18 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:28.960327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5845-efbf-4c69-9130-b12cee97c78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:28.961003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5845-efbf-4c69-9130-b12cee97c78e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.961297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5845-efbf-4c69-9130-b12cee97c78e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.961611 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce5845-efbf-4c69-9130-b12cee97c78e None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.962326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 40/78] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:13:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:28.962418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce5845-efbf-4c69-9130-b12cee97c78e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:28.962701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce5845-efbf-4c69-9130-b12cee97c78e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:28.964134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce5845-efbf-4c69-9130-b12cee97c78e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:28.966698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4b43-1481-4494-9508-900862e242d4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:28.966994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4b43-1481-4494-9508-900862e242d4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:28.967279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cc4b43-1481-4494-9508-900862e242d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:28.972546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 39/79] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:28.977006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5845-efbf-4c69-9130-b12cee97c78e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:28.983143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-99cc4b43-1481-4494-9508-900862e242d4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:28.984333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-99cc4b43-1481-4494-9508-900862e242d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:29.050399 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 41/80] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:29.051231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4b43-1481-4494-9508-900862e242d4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.074840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce5845-efbf-4c69-9130-b12cee97c78e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.082136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:29.107580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33576d6-c6ff-4b56-baca-a5e719d6410a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.108364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33576d6-c6ff-4b56-baca-a5e719d6410a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.108663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33576d6-c6ff-4b56-baca-a5e719d6410a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.108949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33576d6-c6ff-4b56-baca-a5e719d6410a None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.109747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33576d6-c6ff-4b56-baca-a5e719d6410a None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:29.110062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33576d6-c6ff-4b56-baca-a5e719d6410a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.112483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33576d6-c6ff-4b56-baca-a5e719d6410a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.124102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 40/81] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:29.124102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33576d6-c6ff-4b56-baca-a5e719d6410a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.159803 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cc4b43-1481-4494-9508-900862e242d4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.168457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:29.195352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb06a-0edd-4271-9277-2556340f944f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.196221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb06a-0edd-4271-9277-2556340f944f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.196515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb06a-0edd-4271-9277-2556340f944f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.196803 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ccb06a-0edd-4271-9277-2556340f944f None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.198013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccb06a-0edd-4271-9277-2556340f944f None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.199485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ccb06a-0edd-4271-9277-2556340f944f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.201369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 42/82] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:29 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:29.202063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb06a-0edd-4271-9277-2556340f944f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.225800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33576d6-c6ff-4b56-baca-a5e719d6410a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.260109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c6f43-2ddb-411f-aec9-f156c7c046b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.260993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c6f43-2ddb-411f-aec9-f156c7c046b2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.261285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c6f43-2ddb-411f-aec9-f156c7c046b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.261588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c6f43-2ddb-411f-aec9-f156c7c046b2 None admin] PATH_INFO: `/v3/users/4bd8fa54875c432abf00243484dacb64` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.262838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c6f43-2ddb-411f-aec9-f156c7c046b2 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd8fa54875c432abf00243484dacb64)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.264297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938c6f43-2ddb-411f-aec9-f156c7c046b2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.271199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 41/83] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:29 2022] GET /identity/v3/users/4bd8fa54875c432abf00243484dacb64 => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:29.271199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c6f43-2ddb-411f-aec9-f156c7c046b2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.302559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:29.306811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ccb06a-0edd-4271-9277-2556340f944f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.341276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a840d8-fb05-4821-8910-2e97e44baa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.342174 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a840d8-fb05-4821-8910-2e97e44baa67 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.342468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a840d8-fb05-4821-8910-2e97e44baa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.342747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a840d8-fb05-4821-8910-2e97e44baa67 None admin] PATH_INFO: `/v3/users/417a441bd00d4df89cd837724e786c1e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.343938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a840d8-fb05-4821-8910-2e97e44baa67 None admin] RBAC: Authorizing `identity:get_user(user_id=417a441bd00d4df89cd837724e786c1e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.345966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a840d8-fb05-4821-8910-2e97e44baa67 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.347583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 43/84] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:29 2022] GET /identity/v3/users/417a441bd00d4df89cd837724e786c1e => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:29.348339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a840d8-fb05-4821-8910-2e97e44baa67 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.370101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c6f43-2ddb-411f-aec9-f156c7c046b2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.383156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:29.403942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fa9a2-ffa2-4872-8e5e-a76acbbd06b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.404815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fa9a2-ffa2-4872-8e5e-a76acbbd06b0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.405107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fa9a2-ffa2-4872-8e5e-a76acbbd06b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.405408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8fa9a2-ffa2-4872-8e5e-a76acbbd06b0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.406778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fa9a2-ffa2-4872-8e5e-a76acbbd06b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.408710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8fa9a2-ffa2-4872-8e5e-a76acbbd06b0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.410357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 42/85] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:29.411109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fa9a2-ffa2-4872-8e5e-a76acbbd06b0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.448198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a840d8-fb05-4821-8910-2e97e44baa67 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.484629 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62fad4-1c7c-4fc5-980b-94251e335a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.485644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62fad4-1c7c-4fc5-980b-94251e335a58 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.485932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62fad4-1c7c-4fc5-980b-94251e335a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.486208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62fad4-1c7c-4fc5-980b-94251e335a58 None admin] PATH_INFO: `/v3/users/4bd8fa54875c432abf00243484dacb64` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.487408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62fad4-1c7c-4fc5-980b-94251e335a58 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd8fa54875c432abf00243484dacb64)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.488843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62fad4-1c7c-4fc5-980b-94251e335a58 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.490491 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 44/86] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:29 2022] GET /identity/v3/users/4bd8fa54875c432abf00243484dacb64 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:29.491250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62fad4-1c7c-4fc5-980b-94251e335a58 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.514517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8fa9a2-ffa2-4872-8e5e-a76acbbd06b0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.526602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:29.548360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12381a-265a-4a43-aabb-253953b3645d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.549022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12381a-265a-4a43-aabb-253953b3645d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.549314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12381a-265a-4a43-aabb-253953b3645d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.549629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd12381a-265a-4a43-aabb-253953b3645d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.550472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12381a-265a-4a43-aabb-253953b3645d None admin] RBAC: Adding query filter params (role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=23a2e92a012a4c6995bc96295899d286, user.id=4bd8fa54875c432abf00243484dacb64) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:29.550790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12381a-265a-4a43-aabb-253953b3645d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.552243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd12381a-265a-4a43-aabb-253953b3645d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.559968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 43/87] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:29 2022] GET /identity/v3/role_assignments?user.id=4bd8fa54875c432abf00243484dacb64&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:29.561016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12381a-265a-4a43-aabb-253953b3645d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.598515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62fad4-1c7c-4fc5-980b-94251e335a58 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.622451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:29.637872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3538401-7cb9-46e5-9b70-45f77516045e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.638584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3538401-7cb9-46e5-9b70-45f77516045e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.638860 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3538401-7cb9-46e5-9b70-45f77516045e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.639095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3538401-7cb9-46e5-9b70-45f77516045e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.689755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd12381a-265a-4a43-aabb-253953b3645d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.720341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3538401-7cb9-46e5-9b70-45f77516045e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.721882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3538401-7cb9-46e5-9b70-45f77516045e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.726630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f2824-041f-4be1-85d9-2101fd3bd781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.727292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f2824-041f-4be1-85d9-2101fd3bd781 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.727582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f2824-041f-4be1-85d9-2101fd3bd781 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.727864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f2824-041f-4be1-85d9-2101fd3bd781 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.759216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:29.803089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f2824-041f-4be1-85d9-2101fd3bd781 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.804568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f2824-041f-4be1-85d9-2101fd3bd781 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.833842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 45/88] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:29.834842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3538401-7cb9-46e5-9b70-45f77516045e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.851849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:29.913439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 44/89] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:29.914469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f2824-041f-4be1-85d9-2101fd3bd781 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:29.944394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3538401-7cb9-46e5-9b70-45f77516045e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:29.980908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f50c5d-2e61-41f0-9048-39f2f0fcb896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:29.981540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f50c5d-2e61-41f0-9048-39f2f0fcb896 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:29.981831 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f50c5d-2e61-41f0-9048-39f2f0fcb896 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:29.982113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f50c5d-2e61-41f0-9048-39f2f0fcb896 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:29.982840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f50c5d-2e61-41f0-9048-39f2f0fcb896 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:29.984336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f50c5d-2e61-41f0-9048-39f2f0fcb896 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:29.991594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:30.008833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 46/90] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:13:29 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:30.009782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f50c5d-2e61-41f0-9048-39f2f0fcb896 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:30.038828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f2824-041f-4be1-85d9-2101fd3bd781 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:30.097795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ad780-4222-4ef3-b420-189152c9169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:30.099162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ad780-4222-4ef3-b420-189152c9169d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:30.100157 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ad780-4222-4ef3-b420-189152c9169d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:30.100928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ad780-4222-4ef3-b420-189152c9169d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:30.128736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:30.154064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f50c5d-2e61-41f0-9048-39f2f0fcb896 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:30.202168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3302fb-6953-4483-97aa-299ae8fc65a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:30.203055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3302fb-6953-4483-97aa-299ae8fc65a5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:30.203375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3302fb-6953-4483-97aa-299ae8fc65a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:30.203631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3302fb-6953-4483-97aa-299ae8fc65a5 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:30.205597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3302fb-6953-4483-97aa-299ae8fc65a5 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:30.207059 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3302fb-6953-4483-97aa-299ae8fc65a5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:30.208783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 47/91] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:30 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:30.210732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3302fb-6953-4483-97aa-299ae8fc65a5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:30.213845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ad780-4222-4ef3-b420-189152c9169d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:30.215360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ad780-4222-4ef3-b420-189152c9169d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:30.287308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:30.345274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3302fb-6953-4483-97aa-299ae8fc65a5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:30.357922 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 45/92] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:30.359163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ad780-4222-4ef3-b420-189152c9169d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:30.394493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c6f8-da21-44a4-b3ad-c13cf8d02806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:30.396012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c6f8-da21-44a4-b3ad-c13cf8d02806 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:30.396621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c6f8-da21-44a4-b3ad-c13cf8d02806 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:30.396917 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d1c6f8-da21-44a4-b3ad-c13cf8d02806 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:30.398364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1c6f8-da21-44a4-b3ad-c13cf8d02806 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:30.400409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d1c6f8-da21-44a4-b3ad-c13cf8d02806 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:30.402515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 48/93] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:30 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:30.403930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c6f8-da21-44a4-b3ad-c13cf8d02806 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:30.416667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:30.487519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ad780-4222-4ef3-b420-189152c9169d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:30.523376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2335edd-cbce-4aa7-b13a-fc42983c9e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:30.523967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2335edd-cbce-4aa7-b13a-fc42983c9e22 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:30.524256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2335edd-cbce-4aa7-b13a-fc42983c9e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:30.524541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2335edd-cbce-4aa7-b13a-fc42983c9e22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:30.525411 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2335edd-cbce-4aa7-b13a-fc42983c9e22 None admin] RBAC: Adding query filter params (role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=23a2e92a012a4c6995bc96295899d286, user.id=4bd8fa54875c432abf00243484dacb64) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:30.525740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2335edd-cbce-4aa7-b13a-fc42983c9e22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:30.527221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2335edd-cbce-4aa7-b13a-fc42983c9e22 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:30.533866 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 46/94] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:30 2022] GET /identity/v3/role_assignments?user.id=4bd8fa54875c432abf00243484dacb64&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=3b6f494c47a6407284d1d344940e34cd => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:30.534766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2335edd-cbce-4aa7-b13a-fc42983c9e22 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:30.535042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d1c6f8-da21-44a4-b3ad-c13cf8d02806 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:30.566453 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:30.577033 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17025d50-fa6b-4283-b5e8-aac5ff7bc2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:30.577975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025d50-fa6b-4283-b5e8-aac5ff7bc2dd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:30.578435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025d50-fa6b-4283-b5e8-aac5ff7bc2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:30.578733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025d50-fa6b-4283-b5e8-aac5ff7bc2dd None admin] PATH_INFO: `/v3/projects/937026fc7c6144eeb87b6e855519de7f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:30.580781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17025d50-fa6b-4283-b5e8-aac5ff7bc2dd None admin] RBAC: Authorizing `identity:get_project(project_id=937026fc7c6144eeb87b6e855519de7f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:30.582434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17025d50-fa6b-4283-b5e8-aac5ff7bc2dd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:30.584414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 49/95] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:30 2022] GET /identity/v3/projects/937026fc7c6144eeb87b6e855519de7f => generated 312 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:30.590427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17025d50-fa6b-4283-b5e8-aac5ff7bc2dd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:30.663379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2335edd-cbce-4aa7-b13a-fc42983c9e22 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:30.676518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:30.709622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a973386-56a3-4ab0-8041-6484a71db7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:30.710302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a973386-56a3-4ab0-8041-6484a71db7bd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:30.710594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a973386-56a3-4ab0-8041-6484a71db7bd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:30.710877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a973386-56a3-4ab0-8041-6484a71db7bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:30.711738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a973386-56a3-4ab0-8041-6484a71db7bd None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=23a2e92a012a4c6995bc96295899d286, user.id=417a441bd00d4df89cd837724e786c1e) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:30.712058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a973386-56a3-4ab0-8041-6484a71db7bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:30.713545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17025d50-fa6b-4283-b5e8-aac5ff7bc2dd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:30.714051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a973386-56a3-4ab0-8041-6484a71db7bd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:30.719740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 47/96] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:30 2022] GET /identity/v3/role_assignments?user.id=417a441bd00d4df89cd837724e786c1e&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:30.755904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1b2e11-4573-4297-a579-0e763c6e32ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:30.756604 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b2e11-4573-4297-a579-0e763c6e32ff None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:30.756896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b2e11-4573-4297-a579-0e763c6e32ff None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:30.757244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1b2e11-4573-4297-a579-0e763c6e32ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:30.758244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b2e11-4573-4297-a579-0e763c6e32ff None admin] RBAC: Adding query filter params (role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=937026fc7c6144eeb87b6e855519de7f, user.id=4bd8fa54875c432abf00243484dacb64) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:30.758572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b2e11-4573-4297-a579-0e763c6e32ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:30.760126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1b2e11-4573-4297-a579-0e763c6e32ff None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:30.767762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 50/97] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:30 2022] GET /identity/v3/role_assignments?user.id=4bd8fa54875c432abf00243484dacb64&scope.project.id=937026fc7c6144eeb87b6e855519de7f&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:32.900511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0154f-a79a-4851-9959-328cba9b6eac None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:32.901546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0154f-a79a-4851-9959-328cba9b6eac None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:32.902104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0154f-a79a-4851-9959-328cba9b6eac None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:32.903703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 48/98] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:32.911711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3559e26-6ae0-4575-b383-2fb56e2b5de2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:32.912604 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3559e26-6ae0-4575-b383-2fb56e2b5de2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:32.913098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3559e26-6ae0-4575-b383-2fb56e2b5de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:32.937913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c3559e26-6ae0-4575-b383-2fb56e2b5de2 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:32.940148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c3559e26-6ae0-4575-b383-2fb56e2b5de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:33.008434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:33.023737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 51/99] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:33.063144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0154f-a79a-4851-9959-328cba9b6eac None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:33.189059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd0154f-a79a-4851-9959-328cba9b6eac None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:33.233033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557aa59-b598-457b-87c1-1600b1727ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:33.233882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557aa59-b598-457b-87c1-1600b1727ecd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.234180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557aa59-b598-457b-87c1-1600b1727ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.234466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557aa59-b598-457b-87c1-1600b1727ecd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.251557 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557aa59-b598-457b-87c1-1600b1727ecd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5557aa59-b598-457b-87c1-1600b1727ecd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:33.259623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:33.260459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:33.261651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 49/100] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:33.267461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3559e26-6ae0-4575-b383-2fb56e2b5de2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:33.308590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ab793-303f-4aa5-996e-88d6fe80b4be None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.309219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ab793-303f-4aa5-996e-88d6fe80b4be None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.309837 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ab793-303f-4aa5-996e-88d6fe80b4be None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.311676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 50/101] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:33.318285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbef8b-0fc8-4717-b4ef-bb64cc40068b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.318834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbef8b-0fc8-4717-b4ef-bb64cc40068b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.319318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dbef8b-0fc8-4717-b4ef-bb64cc40068b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.342195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-25dbef8b-0fc8-4717-b4ef-bb64cc40068b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:33.343730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-25dbef8b-0fc8-4717-b4ef-bb64cc40068b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:33.391563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:33.430574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3559e26-6ae0-4575-b383-2fb56e2b5de2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:33.473975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 51/102] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:33.475182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913172c-b210-4095-a89a-1e25f230c1c8 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.475679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913172c-b210-4095-a89a-1e25f230c1c8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.476187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0913172c-b210-4095-a89a-1e25f230c1c8 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.477637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 52/103] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:33.478073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3556abcd-c8aa-419f-a150-1954f3b798e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:33.478921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd0df-5f98-47b2-a4e4-a70d89081a1b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.479207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556abcd-c8aa-419f-a150-1954f3b798e3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.479304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd0df-5f98-47b2-a4e4-a70d89081a1b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.479495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1fd0df-5f98-47b2-a4e4-a70d89081a1b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.479943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556abcd-c8aa-419f-a150-1954f3b798e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.480404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3556abcd-c8aa-419f-a150-1954f3b798e3 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.480956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 53/104] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:33.481589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556abcd-c8aa-419f-a150-1954f3b798e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:33.482088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556abcd-c8aa-419f-a150-1954f3b798e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:33.483678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3556abcd-c8aa-419f-a150-1954f3b798e3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:33.487696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b82d4-ced9-41d8-807f-68b86a1a8295 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.487998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b82d4-ced9-41d8-807f-68b86a1a8295 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.488289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1b82d4-ced9-41d8-807f-68b86a1a8295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.494494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 52/105] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:33.496134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc630a-5653-4ca3-893b-96cc3a4760d8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.496724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc630a-5653-4ca3-893b-96cc3a4760d8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.497214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bc630a-5653-4ca3-893b-96cc3a4760d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.507769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0d1b82d4-ced9-41d8-807f-68b86a1a8295 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:33.509015 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0d1b82d4-ced9-41d8-807f-68b86a1a8295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:33.515618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-58bc630a-5653-4ca3-893b-96cc3a4760d8 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:33.517701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-58bc630a-5653-4ca3-893b-96cc3a4760d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:33.584673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 54/106] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:33.585952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b82d4-ced9-41d8-807f-68b86a1a8295 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:33.587781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:33.619250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 53/107] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:33.620415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc630a-5653-4ca3-893b-96cc3a4760d8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:33.723287 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:33.736932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1b82d4-ced9-41d8-807f-68b86a1a8295 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:33.754942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bc630a-5653-4ca3-893b-96cc3a4760d8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:33.781573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54d6c8-f830-445c-82bd-d78412301681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:33.783466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54d6c8-f830-445c-82bd-d78412301681 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.784328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54d6c8-f830-445c-82bd-d78412301681 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.784920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54d6c8-f830-445c-82bd-d78412301681 None admin] PATH_INFO: `/v3/users/4bd8fa54875c432abf00243484dacb64` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.786690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54d6c8-f830-445c-82bd-d78412301681 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd8fa54875c432abf00243484dacb64)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:33.789044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54d6c8-f830-445c-82bd-d78412301681 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:33.790769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 55/108] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:33 2022] GET /identity/v3/users/4bd8fa54875c432abf00243484dacb64 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:33.791550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54d6c8-f830-445c-82bd-d78412301681 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:33.792616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1540-1011-4df2-8e38-307c885125fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:33.793692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1540-1011-4df2-8e38-307c885125fc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.794278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1540-1011-4df2-8e38-307c885125fc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.794748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7c1540-1011-4df2-8e38-307c885125fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.807286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7c1540-1011-4df2-8e38-307c885125fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ea7c1540-1011-4df2-8e38-307c885125fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:33.816415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:33.817308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:33.818080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:33.820622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 54/109] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:33.821660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1540-1011-4df2-8e38-307c885125fc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:33.869763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:33.921583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54d6c8-f830-445c-82bd-d78412301681 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:33.955090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7c1540-1011-4df2-8e38-307c885125fc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:33.960474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461182e-737c-4cec-9178-a99669c6c067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:33.961278 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461182e-737c-4cec-9178-a99669c6c067 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.961591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461182e-737c-4cec-9178-a99669c6c067 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.961873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461182e-737c-4cec-9178-a99669c6c067 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.973418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461182e-737c-4cec-9178-a99669c6c067 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3461182e-737c-4cec-9178-a99669c6c067 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:33.981350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:33.982367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:33.983105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 56/110] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:33.986587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461182e-737c-4cec-9178-a99669c6c067 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:33.991506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9972158-76f3-4623-baf1-bb0a4a706359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:33.995597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9972158-76f3-4623-baf1-bb0a4a706359 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:33.995597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9972158-76f3-4623-baf1-bb0a4a706359 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:33.995597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9972158-76f3-4623-baf1-bb0a4a706359 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:33.995597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9972158-76f3-4623-baf1-bb0a4a706359 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:33.995597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9972158-76f3-4623-baf1-bb0a4a706359 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:33.997047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 55/111] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:33.997836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9972158-76f3-4623-baf1-bb0a4a706359 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:34.011156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:34.092181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461182e-737c-4cec-9178-a99669c6c067 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.098948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9972158-76f3-4623-baf1-bb0a4a706359 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.107496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:34.127141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99750544-8487-4f47-b6ed-e8763b3e4d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:34.128065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99750544-8487-4f47-b6ed-e8763b3e4d48 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.128363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99750544-8487-4f47-b6ed-e8763b3e4d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.128678 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99750544-8487-4f47-b6ed-e8763b3e4d48 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.129969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99750544-8487-4f47-b6ed-e8763b3e4d48 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:34.131570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99750544-8487-4f47-b6ed-e8763b3e4d48 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:34.132934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79095a7-6b12-4133-95fc-2003427d659a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:34.133293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 57/112] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:33 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:34.134644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bacdb-6b03-480d-bd9a-19f2671723f6 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.134941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bacdb-6b03-480d-bd9a-19f2671723f6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.135222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481bacdb-6b03-480d-bd9a-19f2671723f6 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.136454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 58/113] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:34.137578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee0581-61dd-4c0b-878d-649b2157be2f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.138000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee0581-61dd-4c0b-878d-649b2157be2f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.138096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79095a7-6b12-4133-95fc-2003427d659a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.138287 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee0581-61dd-4c0b-878d-649b2157be2f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.138713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79095a7-6b12-4133-95fc-2003427d659a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.139004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79095a7-6b12-4133-95fc-2003427d659a None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.141845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 59/114] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:34.141959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79095a7-6b12-4133-95fc-2003427d659a None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:34.141959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79095a7-6b12-4133-95fc-2003427d659a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:34.142081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee0581-61dd-4c0b-878d-649b2157be2f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:34.142145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79095a7-6b12-4133-95fc-2003427d659a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:34.155263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 56/115] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:34.155263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79095a7-6b12-4133-95fc-2003427d659a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:34.233413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:34.250434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee0581-61dd-4c0b-878d-649b2157be2f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.253482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79095a7-6b12-4133-95fc-2003427d659a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.286003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43113a9a-1990-415b-807d-24d573923665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:34.286673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43113a9a-1990-415b-807d-24d573923665 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.286966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43113a9a-1990-415b-807d-24d573923665 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.287250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43113a9a-1990-415b-807d-24d573923665 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.287421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed359f6a-3e7e-4a65-bcc2-5095831c253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:34.288000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43113a9a-1990-415b-807d-24d573923665 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:34.288088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359f6a-3e7e-4a65-bcc2-5095831c253a None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.288314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43113a9a-1990-415b-807d-24d573923665 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:34.288399 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359f6a-3e7e-4a65-bcc2-5095831c253a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.288636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed359f6a-3e7e-4a65-bcc2-5095831c253a None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.289347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed359f6a-3e7e-4a65-bcc2-5095831c253a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:34.289763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43113a9a-1990-415b-807d-24d573923665 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:34.290817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed359f6a-3e7e-4a65-bcc2-5095831c253a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:34.303182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 60/116] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:34.303182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43113a9a-1990-415b-807d-24d573923665 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:34.316494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 57/117] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:13:34 2022] POST /identity/v3/projects => generated 302 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:34.318044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f37a-5812-4449-b75d-4f3bd3da0596 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.318343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f37a-5812-4449-b75d-4f3bd3da0596 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.318626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769f37a-5812-4449-b75d-4f3bd3da0596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.338288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1769f37a-5812-4449-b75d-4f3bd3da0596 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:34.340131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1769f37a-5812-4449-b75d-4f3bd3da0596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:34.360683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:34.428585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 58/118] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:34.428585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9c163-4d70-4109-896c-569505669555 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.428829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43113a9a-1990-415b-807d-24d573923665 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.428901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9c163-4d70-4109-896c-569505669555 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.428901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa9c163-4d70-4109-896c-569505669555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.448518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9fa9c163-4d70-4109-896c-569505669555 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:34.449838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9fa9c163-4d70-4109-896c-569505669555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:34.477226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bb322-57b9-4db1-b684-fd3b2c3639b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:34.478182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bb322-57b9-4db1-b684-fd3b2c3639b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.478475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bb322-57b9-4db1-b684-fd3b2c3639b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.478761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174bb322-57b9-4db1-b684-fd3b2c3639b6 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286/users/4bd8fa54875c432abf00243484dacb64/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.480749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bb322-57b9-4db1-b684-fd3b2c3639b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=23a2e92a012a4c6995bc96295899d286, user_id=4bd8fa54875c432abf00243484dacb64, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:34.482314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174bb322-57b9-4db1-b684-fd3b2c3639b6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:34.499474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 61/119] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:13:34 2022] PUT /identity/v3/projects/23a2e92a012a4c6995bc96295899d286/users/4bd8fa54875c432abf00243484dacb64/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:13:34.500128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bb322-57b9-4db1-b684-fd3b2c3639b6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:34.518672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:34.553414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 59/120] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:34.554132 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9c163-4d70-4109-896c-569505669555 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:34.636928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174bb322-57b9-4db1-b684-fd3b2c3639b6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.669288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:34.676819 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3a18-c5ab-4cce-88d9-abce2b079cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:34.677753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3a18-c5ab-4cce-88d9-abce2b079cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.678042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3a18-c5ab-4cce-88d9-abce2b079cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.678322 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce3a18-c5ab-4cce-88d9-abce2b079cf7 None admin] PATH_INFO: `/v3/users/4bd8fa54875c432abf00243484dacb64` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.681130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce3a18-c5ab-4cce-88d9-abce2b079cf7 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd8fa54875c432abf00243484dacb64)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:34.682666 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce3a18-c5ab-4cce-88d9-abce2b079cf7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:34.684593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 62/121] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:34 2022] GET /identity/v3/users/4bd8fa54875c432abf00243484dacb64 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:34.685101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3a18-c5ab-4cce-88d9-abce2b079cf7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:34.697022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa9c163-4d70-4109-896c-569505669555 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.746351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904fdf-c8c2-4628-b82d-88aef62441c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:34.747499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904fdf-c8c2-4628-b82d-88aef62441c0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.747791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904fdf-c8c2-4628-b82d-88aef62441c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.748480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3904fdf-c8c2-4628-b82d-88aef62441c0 None admin] PATH_INFO: `/v3/users/6277667b84b54cbc9a6cc4d5288e746a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.767623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3904fdf-c8c2-4628-b82d-88aef62441c0 None admin] RBAC: Authorizing `identity:get_user(user_id=6277667b84b54cbc9a6cc4d5288e746a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:34.770142 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3904fdf-c8c2-4628-b82d-88aef62441c0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:34.771891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 60/122] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:34 2022] GET /identity/v3/users/6277667b84b54cbc9a6cc4d5288e746a => generated 289 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:34.772667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904fdf-c8c2-4628-b82d-88aef62441c0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:34.784764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:34.812284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce3a18-c5ab-4cce-88d9-abce2b079cf7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.855298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e811b1-2c04-4b30-a3e6-93a79bebf2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:34.856104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e811b1-2c04-4b30-a3e6-93a79bebf2cd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.856396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e811b1-2c04-4b30-a3e6-93a79bebf2cd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.856679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e811b1-2c04-4b30-a3e6-93a79bebf2cd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.872906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e811b1-2c04-4b30-a3e6-93a79bebf2cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-80e811b1-2c04-4b30-a3e6-93a79bebf2cd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:34.881548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:34.882468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:34.883240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 63/123] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:13:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:34.883853 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e811b1-2c04-4b30-a3e6-93a79bebf2cd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:34.918352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3904fdf-c8c2-4628-b82d-88aef62441c0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.965963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceebed9-72c2-4360-ae14-a67f0357633f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:34.967480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceebed9-72c2-4360-ae14-a67f0357633f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:34.968421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceebed9-72c2-4360-ae14-a67f0357633f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:34.968849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceebed9-72c2-4360-ae14-a67f0357633f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:34.979608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:34.984095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceebed9-72c2-4360-ae14-a67f0357633f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:34.995309 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e811b1-2c04-4b30-a3e6-93a79bebf2cd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-1ceebed9-72c2-4360-ae14-a67f0357633f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:34.997417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:34.998433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:34.999404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:35.008026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 61/124] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:13:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:35.009127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceebed9-72c2-4360-ae14-a67f0357633f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:35.042780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fc87f-d2c7-4208-97ea-61c0318dcdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:35.044495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fc87f-d2c7-4208-97ea-61c0318dcdb9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:35.044942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fc87f-d2c7-4208-97ea-61c0318dcdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:35.045364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19fc87f-d2c7-4208-97ea-61c0318dcdb9 None admin] PATH_INFO: `/v3/projects/937026fc7c6144eeb87b6e855519de7f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:35.046955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fc87f-d2c7-4208-97ea-61c0318dcdb9 None admin] RBAC: Authorizing `identity:get_project(project_id=937026fc7c6144eeb87b6e855519de7f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:35.050011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19fc87f-d2c7-4208-97ea-61c0318dcdb9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:35.051876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 64/125] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:34 2022] GET /identity/v3/projects/937026fc7c6144eeb87b6e855519de7f => generated 312 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:35.057238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fc87f-d2c7-4208-97ea-61c0318dcdb9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:35.070825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:35.123001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceebed9-72c2-4360-ae14-a67f0357633f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:35.167434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de894922-14b0-413a-a344-7c3220df0a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:35.168588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de894922-14b0-413a-a344-7c3220df0a42 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:35.169046 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de894922-14b0-413a-a344-7c3220df0a42 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:35.169500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de894922-14b0-413a-a344-7c3220df0a42 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:35.183106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19fc87f-d2c7-4208-97ea-61c0318dcdb9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:35.228261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948de22f-54b4-4f4f-87f5-2c0cfd2bc374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:35.229167 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948de22f-54b4-4f4f-87f5-2c0cfd2bc374 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:35.230208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948de22f-54b4-4f4f-87f5-2c0cfd2bc374 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:35.230637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948de22f-54b4-4f4f-87f5-2c0cfd2bc374 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:35.231802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948de22f-54b4-4f4f-87f5-2c0cfd2bc374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:35.232255 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948de22f-54b4-4f4f-87f5-2c0cfd2bc374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:35.234458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948de22f-54b4-4f4f-87f5-2c0cfd2bc374 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:35.246030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:35.248050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 65/126] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:35.250281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948de22f-54b4-4f4f-87f5-2c0cfd2bc374 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:35.254018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de894922-14b0-413a-a344-7c3220df0a42 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:35.255745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de894922-14b0-413a-a344-7c3220df0a42 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:35.348823 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:35.371101 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 62/127] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:35.376789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de894922-14b0-413a-a344-7c3220df0a42 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:35.391213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948de22f-54b4-4f4f-87f5-2c0cfd2bc374 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:35.433688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4bdac-460f-43c8-8713-a26e219bb6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:35.434731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4bdac-460f-43c8-8713-a26e219bb6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:35.435219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4bdac-460f-43c8-8713-a26e219bb6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:35.435971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4bdac-460f-43c8-8713-a26e219bb6d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:35.436920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4bdac-460f-43c8-8713-a26e219bb6d7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:35.437594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4bdac-460f-43c8-8713-a26e219bb6d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:35.439363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed4bdac-460f-43c8-8713-a26e219bb6d7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:35.448024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 66/128] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:13:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:35.453455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4bdac-460f-43c8-8713-a26e219bb6d7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:35.504946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:35.509137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de894922-14b0-413a-a344-7c3220df0a42 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:35.554116 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682bacd-328a-41af-bcdb-88195c63fca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:35.555179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682bacd-328a-41af-bcdb-88195c63fca1 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:35.555561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682bacd-328a-41af-bcdb-88195c63fca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:35.555936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682bacd-328a-41af-bcdb-88195c63fca1 None admin] PATH_INFO: `/v3/projects/937026fc7c6144eeb87b6e855519de7f/users/4bd8fa54875c432abf00243484dacb64/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:35.558785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682bacd-328a-41af-bcdb-88195c63fca1 None admin] RBAC: Authorizing `identity:create_grant(project_id=937026fc7c6144eeb87b6e855519de7f, user_id=4bd8fa54875c432abf00243484dacb64, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:35.561271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6682bacd-328a-41af-bcdb-88195c63fca1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:35.585496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 63/129] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:13:35 2022] PUT /identity/v3/projects/937026fc7c6144eeb87b6e855519de7f/users/4bd8fa54875c432abf00243484dacb64/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:13:35.586403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682bacd-328a-41af-bcdb-88195c63fca1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:35.593046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4bdac-460f-43c8-8713-a26e219bb6d7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:35.660101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:35.670335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cf121-5179-4a94-a2e6-d79ca51aee5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:35.671311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cf121-5179-4a94-a2e6-d79ca51aee5f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:35.671603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cf121-5179-4a94-a2e6-d79ca51aee5f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:35.671887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4cf121-5179-4a94-a2e6-d79ca51aee5f None admin] PATH_INFO: `/v3/users/417a441bd00d4df89cd837724e786c1e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:35.678819 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cf121-5179-4a94-a2e6-d79ca51aee5f None admin] RBAC: Authorizing `identity:get_user(user_id=417a441bd00d4df89cd837724e786c1e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:35.680354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4cf121-5179-4a94-a2e6-d79ca51aee5f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:35.682235 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 67/130] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:35 2022] GET /identity/v3/users/417a441bd00d4df89cd837724e786c1e => generated 247 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:35.686456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cf121-5179-4a94-a2e6-d79ca51aee5f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:35.760661 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682bacd-328a-41af-bcdb-88195c63fca1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:35.800426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045570f0-f214-4c37-8626-01f330ee8426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:35.801306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045570f0-f214-4c37-8626-01f330ee8426 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:35.801626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045570f0-f214-4c37-8626-01f330ee8426 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:35.801908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045570f0-f214-4c37-8626-01f330ee8426 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:35.803116 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045570f0-f214-4c37-8626-01f330ee8426 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:35.804571 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045570f0-f214-4c37-8626-01f330ee8426 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:35.806412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 64/131] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:35 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:35.807197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045570f0-f214-4c37-8626-01f330ee8426 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:35.832091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4cf121-5179-4a94-a2e6-d79ca51aee5f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:35.845146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:35.877197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb816a-f86f-4bf1-ae50-a192e042f63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:35.878442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb816a-f86f-4bf1-ae50-a192e042f63d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:35.878865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb816a-f86f-4bf1-ae50-a192e042f63d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:35.879263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb816a-f86f-4bf1-ae50-a192e042f63d None admin] PATH_INFO: `/v3/users/4bd8fa54875c432abf00243484dacb64` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:35.880617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb816a-f86f-4bf1-ae50-a192e042f63d None admin] RBAC: Authorizing `identity:get_user(user_id=4bd8fa54875c432abf00243484dacb64)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:35.882801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb816a-f86f-4bf1-ae50-a192e042f63d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:35.888928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 68/132] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:35 2022] GET /identity/v3/users/4bd8fa54875c432abf00243484dacb64 => generated 275 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:35.889855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb816a-f86f-4bf1-ae50-a192e042f63d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:35.952247 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045570f0-f214-4c37-8626-01f330ee8426 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:36.011285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:36.012741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05636d83-831e-4f60-8077-475c42ebc6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:36.013643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05636d83-831e-4f60-8077-475c42ebc6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:36.013924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05636d83-831e-4f60-8077-475c42ebc6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:36.014203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05636d83-831e-4f60-8077-475c42ebc6f3 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:36.016384 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05636d83-831e-4f60-8077-475c42ebc6f3 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:36.019440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05636d83-831e-4f60-8077-475c42ebc6f3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:36.021334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 65/133] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:35 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:36.023651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05636d83-831e-4f60-8077-475c42ebc6f3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:36.079884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb816a-f86f-4bf1-ae50-a192e042f63d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:36.146162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e100a-13c1-42d5-b2bc-41b3d6371328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:36.146852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e100a-13c1-42d5-b2bc-41b3d6371328 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:36.147150 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e100a-13c1-42d5-b2bc-41b3d6371328 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:36.147446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4e100a-13c1-42d5-b2bc-41b3d6371328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:36.148311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e100a-13c1-42d5-b2bc-41b3d6371328 None admin] RBAC: Adding query filter params (role.id=4f7a6e180d6d454bb3458ca27d7d822f, scope.project.id=23a2e92a012a4c6995bc96295899d286, user.id=6277667b84b54cbc9a6cc4d5288e746a) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:36.148635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e100a-13c1-42d5-b2bc-41b3d6371328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:36.150136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4e100a-13c1-42d5-b2bc-41b3d6371328 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:36.156180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 69/134] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:35 2022] GET /identity/v3/role_assignments?user.id=6277667b84b54cbc9a6cc4d5288e746a&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=4f7a6e180d6d454bb3458ca27d7d822f => generated 259 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:36.156806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e100a-13c1-42d5-b2bc-41b3d6371328 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:36.228742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:36.242134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05636d83-831e-4f60-8077-475c42ebc6f3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:36.313045 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e72517-e87b-40a9-b8e5-eed7573931d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:36.314613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e72517-e87b-40a9-b8e5-eed7573931d8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:36.314900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e72517-e87b-40a9-b8e5-eed7573931d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:36.315180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e72517-e87b-40a9-b8e5-eed7573931d8 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:36.317557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e72517-e87b-40a9-b8e5-eed7573931d8 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:36.319027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e72517-e87b-40a9-b8e5-eed7573931d8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:36.320937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 66/135] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:36 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:36.326543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e72517-e87b-40a9-b8e5-eed7573931d8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:36.343208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4e100a-13c1-42d5-b2bc-41b3d6371328 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:36.401751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:36.430172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e322d68-b1f2-4cb8-b251-d63104ba7fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:36.431094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322d68-b1f2-4cb8-b251-d63104ba7fe4 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:36.431386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322d68-b1f2-4cb8-b251-d63104ba7fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:36.431668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e322d68-b1f2-4cb8-b251-d63104ba7fe4 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286/users/417a441bd00d4df89cd837724e786c1e/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:36.439004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e322d68-b1f2-4cb8-b251-d63104ba7fe4 None admin] RBAC: Authorizing `identity:create_grant(project_id=23a2e92a012a4c6995bc96295899d286, user_id=417a441bd00d4df89cd837724e786c1e, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:36.440607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e322d68-b1f2-4cb8-b251-d63104ba7fe4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:36.462140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e72517-e87b-40a9-b8e5-eed7573931d8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:36.476503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 70/136] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:13:36 2022] PUT /identity/v3/projects/23a2e92a012a4c6995bc96295899d286/users/417a441bd00d4df89cd837724e786c1e/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:13:36.511654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec80111-683e-45ba-a6b2-4a7576590ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:36.512594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec80111-683e-45ba-a6b2-4a7576590ecd None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:36.512885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec80111-683e-45ba-a6b2-4a7576590ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:36.513254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec80111-683e-45ba-a6b2-4a7576590ecd None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286/users/4bd8fa54875c432abf00243484dacb64/roles/3b6f494c47a6407284d1d344940e34cd` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:36.515991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec80111-683e-45ba-a6b2-4a7576590ecd None admin] RBAC: Authorizing `identity:create_grant(project_id=23a2e92a012a4c6995bc96295899d286, user_id=4bd8fa54875c432abf00243484dacb64, role_id=3b6f494c47a6407284d1d344940e34cd)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:36.518322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec80111-683e-45ba-a6b2-4a7576590ecd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:36.536793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 67/137] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:13:36 2022] PUT /identity/v3/projects/23a2e92a012a4c6995bc96295899d286/users/4bd8fa54875c432abf00243484dacb64/roles/3b6f494c47a6407284d1d344940e34cd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:13:36.746765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad63a81-9adf-4419-b91b-791b2e5042ca None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:36.747141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad63a81-9adf-4419-b91b-791b2e5042ca None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:36.747316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad63a81-9adf-4419-b91b-791b2e5042ca None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:36.748833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 71/138] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:36.755045 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492708-3196-4d85-96a7-d22ed8c38030 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:36.755343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492708-3196-4d85-96a7-d22ed8c38030 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:36.755628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63492708-3196-4d85-96a7-d22ed8c38030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:36.777297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-63492708-3196-4d85-96a7-d22ed8c38030 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:36.778811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-63492708-3196-4d85-96a7-d22ed8c38030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:36.799171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:36.858722 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 68/139] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:36.901894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad63a81-9adf-4419-b91b-791b2e5042ca None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:36.963017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65321b64-0fa6-40a4-8f30-7b58069f6a65 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:36.963670 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65321b64-0fa6-40a4-8f30-7b58069f6a65 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:36.964156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65321b64-0fa6-40a4-8f30-7b58069f6a65 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:36.966053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 69/140] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:36.974924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204dab3-23b1-43f9-bd29-ab290a95bc7a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:36.975228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204dab3-23b1-43f9-bd29-ab290a95bc7a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:36.975994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204dab3-23b1-43f9-bd29-ab290a95bc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:37.011153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7204dab3-23b1-43f9-bd29-ab290a95bc7a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:37.012369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7204dab3-23b1-43f9-bd29-ab290a95bc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:37.068541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad63a81-9adf-4419-b91b-791b2e5042ca None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:37.144918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854292c-3f3a-418d-9251-a7862b406f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:37.145766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854292c-3f3a-418d-9251-a7862b406f30 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:37.146055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854292c-3f3a-418d-9251-a7862b406f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:37.146340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854292c-3f3a-418d-9251-a7862b406f30 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:37.147668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854292c-3f3a-418d-9251-a7862b406f30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:37.149139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854292c-3f3a-418d-9251-a7862b406f30 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:37.153252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 72/141] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:37.158774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854292c-3f3a-418d-9251-a7862b406f30 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:37.162720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 70/142] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:37.235815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204dab3-23b1-43f9-bd29-ab290a95bc7a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:37.260291 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:37.354599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854292c-3f3a-418d-9251-a7862b406f30 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:37.421163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9469037-225d-471e-94ca-316fdb935b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:37.421882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9469037-225d-471e-94ca-316fdb935b41 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:37.422176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9469037-225d-471e-94ca-316fdb935b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:37.424023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9469037-225d-471e-94ca-316fdb935b41 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:37.424771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9469037-225d-471e-94ca-316fdb935b41 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:37.426292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9469037-225d-471e-94ca-316fdb935b41 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:37.459336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:37.480119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 73/143] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:13:37 2022] POST /identity/v3/users => generated 283 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:37.496651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204dab3-23b1-43f9-bd29-ab290a95bc7a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:37.559423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92346c9f-c2e0-4334-a3ef-651f7e4d00e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:37.560244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92346c9f-c2e0-4334-a3ef-651f7e4d00e3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:37.560790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92346c9f-c2e0-4334-a3ef-651f7e4d00e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:37.561065 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92346c9f-c2e0-4334-a3ef-651f7e4d00e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:37.577709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92346c9f-c2e0-4334-a3ef-651f7e4d00e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-92346c9f-c2e0-4334-a3ef-651f7e4d00e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:37.586513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:37.587475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:37.588249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 71/144] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:37.593814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9469037-225d-471e-94ca-316fdb935b41 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:37.716005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9469037-225d-471e-94ca-316fdb935b41 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:37.732673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:37.755935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfd786-fd0a-494f-b7e5-139fa70f89c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:37.756633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfd786-fd0a-494f-b7e5-139fa70f89c9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:37.756915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfd786-fd0a-494f-b7e5-139fa70f89c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:37.757213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bfd786-fd0a-494f-b7e5-139fa70f89c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:37.758056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfd786-fd0a-494f-b7e5-139fa70f89c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:37.758863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfd786-fd0a-494f-b7e5-139fa70f89c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:37.759845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bfd786-fd0a-494f-b7e5-139fa70f89c9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:37.773556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 74/145] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:37.778495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92346c9f-c2e0-4334-a3ef-651f7e4d00e3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:37.884886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92346c9f-c2e0-4334-a3ef-651f7e4d00e3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:37.922138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6a204-cb17-4686-bd86-4da36a6e62d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:37.924048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6a204-cb17-4686-bd86-4da36a6e62d5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:37.924550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6a204-cb17-4686-bd86-4da36a6e62d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:37.925012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d6a204-cb17-4686-bd86-4da36a6e62d5 None admin] PATH_INFO: `/v3/users/4bd8fa54875c432abf00243484dacb64` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:37.926453 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6a204-cb17-4686-bd86-4da36a6e62d5 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd8fa54875c432abf00243484dacb64)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:37.928165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d6a204-cb17-4686-bd86-4da36a6e62d5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:37.930453 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 72/146] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:37 2022] GET /identity/v3/users/4bd8fa54875c432abf00243484dacb64 => generated 275 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:37.936382 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfd786-fd0a-494f-b7e5-139fa70f89c9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:38.042006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bfd786-fd0a-494f-b7e5-139fa70f89c9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:38.078793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d0261-a79f-4558-bb96-37720349766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:38.080282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d0261-a79f-4558-bb96-37720349766d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:38.080797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d0261-a79f-4558-bb96-37720349766d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:38.081265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d0261-a79f-4558-bb96-37720349766d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:38.129867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:38.154227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d0261-a79f-4558-bb96-37720349766d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:38.156056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d0261-a79f-4558-bb96-37720349766d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:38.265635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 75/147] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:38.271493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6a204-cb17-4686-bd86-4da36a6e62d5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:38.295548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:38.382589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d6a204-cb17-4686-bd86-4da36a6e62d5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:38.418627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b1ab8f-42f6-4b45-b61c-fbda4cc22bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:38.419510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1ab8f-42f6-4b45-b61c-fbda4cc22bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:38.419797 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1ab8f-42f6-4b45-b61c-fbda4cc22bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:38.420073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b1ab8f-42f6-4b45-b61c-fbda4cc22bd0 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:38.421293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1ab8f-42f6-4b45-b61c-fbda4cc22bd0 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:38.422811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b1ab8f-42f6-4b45-b61c-fbda4cc22bd0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:38.424514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 73/148] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:38 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:38.430282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d0261-a79f-4558-bb96-37720349766d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:38.481901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6192eb-99c9-4d2c-8a26-6ea2ff1928f4 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:38.482468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6192eb-99c9-4d2c-8a26-6ea2ff1928f4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:38.482757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6192eb-99c9-4d2c-8a26-6ea2ff1928f4 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:38.484429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 74/149] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:38.496203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0c3ac-fd6c-4f26-a868-c06bc7291fd2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:38.496489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0c3ac-fd6c-4f26-a868-c06bc7291fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:38.496771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0c3ac-fd6c-4f26-a868-c06bc7291fd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:38.520656 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d5c0c3ac-fd6c-4f26-a868-c06bc7291fd2 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:38.522188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d5c0c3ac-fd6c-4f26-a868-c06bc7291fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:38.531434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:38.563641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d0261-a79f-4558-bb96-37720349766d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:38.603789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 75/150] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:38.609433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5698f7-c0b2-4257-8cd7-6d389fdc7279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:38.609433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5698f7-c0b2-4257-8cd7-6d389fdc7279 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:38.609433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5698f7-c0b2-4257-8cd7-6d389fdc7279 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:38.609433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5698f7-c0b2-4257-8cd7-6d389fdc7279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:38.609757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5698f7-c0b2-4257-8cd7-6d389fdc7279 None admin] RBAC: Adding query filter params (role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=23a2e92a012a4c6995bc96295899d286, user.id=4bd8fa54875c432abf00243484dacb64) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:38.609757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5698f7-c0b2-4257-8cd7-6d389fdc7279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:38.609757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5698f7-c0b2-4257-8cd7-6d389fdc7279 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:38.616456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 76/151] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:38 2022] GET /identity/v3/role_assignments?user.id=4bd8fa54875c432abf00243484dacb64&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:38.629884 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0c3ac-fd6c-4f26-a868-c06bc7291fd2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:38.649447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:38.776552 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0c3ac-fd6c-4f26-a868-c06bc7291fd2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:38.821896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073bc3a-f77a-4144-a273-cd5d1c38587a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:38.822631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073bc3a-f77a-4144-a273-cd5d1c38587a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:38.822915 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073bc3a-f77a-4144-a273-cd5d1c38587a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:38.823192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073bc3a-f77a-4144-a273-cd5d1c38587a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:38.838711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073bc3a-f77a-4144-a273-cd5d1c38587a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-2073bc3a-f77a-4144-a273-cd5d1c38587a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:38.846927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:38.847790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:38.848526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 76/152] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:38.853936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5698f7-c0b2-4257-8cd7-6d389fdc7279 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:38.973692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5698f7-c0b2-4257-8cd7-6d389fdc7279 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:39.013744 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94761b-2cba-4498-bee8-b4db51a20939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:39.014775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94761b-2cba-4498-bee8-b4db51a20939 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.015255 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94761b-2cba-4498-bee8-b4db51a20939 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.015740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d94761b-2cba-4498-bee8-b4db51a20939 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.016783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94761b-2cba-4498-bee8-b4db51a20939 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:39.017294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94761b-2cba-4498-bee8-b4db51a20939 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:39.018940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d94761b-2cba-4498-bee8-b4db51a20939 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:39.029226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 77/153] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:39.035491 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073bc3a-f77a-4144-a273-cd5d1c38587a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:39.150483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:39.191893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073bc3a-f77a-4144-a273-cd5d1c38587a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:39.231654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ae269-d4af-41af-962d-3127f8ec5a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:39.232588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ae269-d4af-41af-962d-3127f8ec5a03 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.232879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ae269-d4af-41af-962d-3127f8ec5a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.233163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3ae269-d4af-41af-962d-3127f8ec5a03 None admin] PATH_INFO: `/v3/users/4bd8fa54875c432abf00243484dacb64` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.234430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ae269-d4af-41af-962d-3127f8ec5a03 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd8fa54875c432abf00243484dacb64)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:39.235983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3ae269-d4af-41af-962d-3127f8ec5a03 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:39.238119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 77/154] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:39 2022] GET /identity/v3/users/4bd8fa54875c432abf00243484dacb64 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:39.243639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94761b-2cba-4498-bee8-b4db51a20939 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:39.249850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:39.382029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d94761b-2cba-4498-bee8-b4db51a20939 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:39.432480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ccf6ed-3df1-49d9-87b1-3c190f06298a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:39.433503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccf6ed-3df1-49d9-87b1-3c190f06298a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.434005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccf6ed-3df1-49d9-87b1-3c190f06298a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.434480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ccf6ed-3df1-49d9-87b1-3c190f06298a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.557126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccf6ed-3df1-49d9-87b1-3c190f06298a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:39.558981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ccf6ed-3df1-49d9-87b1-3c190f06298a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:39.561345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:39.682195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 78/155] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:39.688274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ae269-d4af-41af-962d-3127f8ec5a03 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:39.746693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab3187-0d81-4a01-8a3f-44345cd261bf None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.746988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab3187-0d81-4a01-8a3f-44345cd261bf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.747252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fab3187-0d81-4a01-8a3f-44345cd261bf None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.748902 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 79/156] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:39.757115 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cdc43-44bf-4beb-a13f-d1b9fbfc1619 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.757424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cdc43-44bf-4beb-a13f-d1b9fbfc1619 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.757703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6cdc43-44bf-4beb-a13f-d1b9fbfc1619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.790388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1b6cdc43-44bf-4beb-a13f-d1b9fbfc1619 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:39.791643 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1b6cdc43-44bf-4beb-a13f-d1b9fbfc1619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:39.868374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3ae269-d4af-41af-962d-3127f8ec5a03 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:39.930128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 80/157] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:39.931542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546adbff-b248-46e0-ba39-3a00f9e3acdb None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.931879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546adbff-b248-46e0-ba39-3a00f9e3acdb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.932097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546adbff-b248-46e0-ba39-3a00f9e3acdb None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.932417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6367a-e0d4-451f-944d-7a9646c1ecbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:39.933364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 81/158] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:39.933723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6367a-e0d4-451f-944d-7a9646c1ecbf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.934018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6367a-e0d4-451f-944d-7a9646c1ecbf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.934298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6367a-e0d4-451f-944d-7a9646c1ecbf None admin] PATH_INFO: `/v3/projects/937026fc7c6144eeb87b6e855519de7f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.936273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462f6d3-ecb7-482b-9877-2594c4492e38 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.936772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462f6d3-ecb7-482b-9877-2594c4492e38 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.937246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2462f6d3-ecb7-482b-9877-2594c4492e38 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.938906 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6367a-e0d4-451f-944d-7a9646c1ecbf None admin] RBAC: Authorizing `identity:get_project(project_id=937026fc7c6144eeb87b6e855519de7f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:39.939630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 82/159] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:39.945437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f6367a-e0d4-451f-944d-7a9646c1ecbf None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:39.945437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 78/160] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:39 2022] GET /identity/v3/projects/937026fc7c6144eeb87b6e855519de7f => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:39.947077 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b84daa-96cd-4c43-bdb5-388fde64ea68 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.947491 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b84daa-96cd-4c43-bdb5-388fde64ea68 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.947760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb761f75-294e-4d86-a089-d42fe16c7ef9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:39.947856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b84daa-96cd-4c43-bdb5-388fde64ea68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.948320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb761f75-294e-4d86-a089-d42fe16c7ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:39.948793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb761f75-294e-4d86-a089-d42fe16c7ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:39.968068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:39.974791 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-eb761f75-294e-4d86-a089-d42fe16c7ef9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:39.976226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-eb761f75-294e-4d86-a089-d42fe16c7ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:39.977270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-23b84daa-96cd-4c43-bdb5-388fde64ea68 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:39.978637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-23b84daa-96cd-4c43-bdb5-388fde64ea68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:40.011324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:40.104815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 83/161] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:40.110936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb761f75-294e-4d86-a089-d42fe16c7ef9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:40.118638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 79/162] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:40.119715 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b84daa-96cd-4c43-bdb5-388fde64ea68 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:40.288504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb761f75-294e-4d86-a089-d42fe16c7ef9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:40.332358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b84daa-96cd-4c43-bdb5-388fde64ea68 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:40.357576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e6e92-cbf4-4f79-aada-c1f8e1f64cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:40.358619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e6e92-cbf4-4f79-aada-c1f8e1f64cda None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:40.359048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e6e92-cbf4-4f79-aada-c1f8e1f64cda None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:40.359783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e6e92-cbf4-4f79-aada-c1f8e1f64cda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:40.361646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e6e92-cbf4-4f79-aada-c1f8e1f64cda None admin] RBAC: Adding query filter params (role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=937026fc7c6144eeb87b6e855519de7f, user.id=4bd8fa54875c432abf00243484dacb64) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:40.362965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e6e92-cbf4-4f79-aada-c1f8e1f64cda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:40.365552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5e6e92-cbf4-4f79-aada-c1f8e1f64cda None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:40.379261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 84/163] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:40 2022] GET /identity/v3/role_assignments?user.id=4bd8fa54875c432abf00243484dacb64&scope.project.id=937026fc7c6144eeb87b6e855519de7f&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:40.381253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e6e92-cbf4-4f79-aada-c1f8e1f64cda None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:40.416782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44b468-12dc-47aa-9b03-c45175c065e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:40.417908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44b468-12dc-47aa-9b03-c45175c065e3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:40.418783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44b468-12dc-47aa-9b03-c45175c065e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:40.419891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44b468-12dc-47aa-9b03-c45175c065e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:40.425842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:40.445139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a44b468-12dc-47aa-9b03-c45175c065e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-0a44b468-12dc-47aa-9b03-c45175c065e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:40.455632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:40.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 80/164] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:13:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:40.457422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44b468-12dc-47aa-9b03-c45175c065e3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:40.462448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:40.562830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44b468-12dc-47aa-9b03-c45175c065e3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:40.606765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8266d0d-7e56-4153-ba61-f9c0f1f74940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:40.607591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8266d0d-7e56-4153-ba61-f9c0f1f74940 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:40.607880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8266d0d-7e56-4153-ba61-f9c0f1f74940 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:40.608160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8266d0d-7e56-4153-ba61-f9c0f1f74940 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:40.631782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8266d0d-7e56-4153-ba61-f9c0f1f74940 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:40.639612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e6e92-cbf4-4f79-aada-c1f8e1f64cda None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b8266d0d-7e56-4153-ba61-f9c0f1f74940 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:40.648178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:40.649326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:40.650210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 81/165] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:40.651195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8266d0d-7e56-4153-ba61-f9c0f1f74940 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:40.737837 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab315e6-e851-4f47-a34c-62ad12be5845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:40.738668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab315e6-e851-4f47-a34c-62ad12be5845 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:40.738955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab315e6-e851-4f47-a34c-62ad12be5845 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:40.739233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab315e6-e851-4f47-a34c-62ad12be5845 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:40.746795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:40.765241 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab315e6-e851-4f47-a34c-62ad12be5845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-eab315e6-e851-4f47-a34c-62ad12be5845 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:40.778726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:40.779625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:40.780390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 85/166] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:13:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:40.781046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab315e6-e851-4f47-a34c-62ad12be5845 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:40.833124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8266d0d-7e56-4153-ba61-f9c0f1f74940 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:40.899893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9948cb-ec58-4e56-89fb-63480284bea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:40.900570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9948cb-ec58-4e56-89fb-63480284bea6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:40.900858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9948cb-ec58-4e56-89fb-63480284bea6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:40.901139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9948cb-ec58-4e56-89fb-63480284bea6 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:40.901948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9948cb-ec58-4e56-89fb-63480284bea6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:40.902269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9948cb-ec58-4e56-89fb-63480284bea6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:40.903666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9948cb-ec58-4e56-89fb-63480284bea6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:40.913341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 82/167] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:40.914193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9948cb-ec58-4e56-89fb-63480284bea6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:40.950539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:40.988540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab315e6-e851-4f47-a34c-62ad12be5845 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:41.019323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9948cb-ec58-4e56-89fb-63480284bea6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:41.037777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:41.040948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ed231-6276-4382-ba28-8782d07b71d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:41.041773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ed231-6276-4382-ba28-8782d07b71d0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.042221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ed231-6276-4382-ba28-8782d07b71d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.042616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97ed231-6276-4382-ba28-8782d07b71d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.043655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ed231-6276-4382-ba28-8782d07b71d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:41.044110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ed231-6276-4382-ba28-8782d07b71d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:41.045607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97ed231-6276-4382-ba28-8782d07b71d0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:41.057642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a68cd-f6cf-478b-984c-f4e6320ed0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:41.058368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a68cd-f6cf-478b-984c-f4e6320ed0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.058541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 86/168] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:41.058744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a68cd-f6cf-478b-984c-f4e6320ed0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.058938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a68cd-f6cf-478b-984c-f4e6320ed0b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.059727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a68cd-f6cf-478b-984c-f4e6320ed0b8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:41.060047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a68cd-f6cf-478b-984c-f4e6320ed0b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:41.061785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12a68cd-f6cf-478b-984c-f4e6320ed0b8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:41.069769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ed231-6276-4382-ba28-8782d07b71d0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:41.076890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 83/169] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:13:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:41.077818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a68cd-f6cf-478b-984c-f4e6320ed0b8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:41.254533 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97ed231-6276-4382-ba28-8782d07b71d0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:41.291690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5790cec-8b62-4fb8-b134-4234ddf08cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:41.292992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5790cec-8b62-4fb8-b134-4234ddf08cff None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.293342 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5790cec-8b62-4fb8-b134-4234ddf08cff None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.293653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5790cec-8b62-4fb8-b134-4234ddf08cff None admin] PATH_INFO: `/v3/users/417a441bd00d4df89cd837724e786c1e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.295020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5790cec-8b62-4fb8-b134-4234ddf08cff None admin] RBAC: Authorizing `identity:get_user(user_id=417a441bd00d4df89cd837724e786c1e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:41.296504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5790cec-8b62-4fb8-b134-4234ddf08cff None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:41.299153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 87/170] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:41 2022] GET /identity/v3/users/417a441bd00d4df89cd837724e786c1e => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:41.299931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5790cec-8b62-4fb8-b134-4234ddf08cff None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:41.302532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a68cd-f6cf-478b-984c-f4e6320ed0b8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:41.337750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:41.382648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cb41d-fc90-46de-bf2d-b75b2dddff27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:41.383955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cb41d-fc90-46de-bf2d-b75b2dddff27 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.384437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cb41d-fc90-46de-bf2d-b75b2dddff27 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.384941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cb41d-fc90-46de-bf2d-b75b2dddff27 None admin] PATH_INFO: `/v3/users/6277667b84b54cbc9a6cc4d5288e746a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.386897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cb41d-fc90-46de-bf2d-b75b2dddff27 None admin] RBAC: Authorizing `identity:get_user(user_id=6277667b84b54cbc9a6cc4d5288e746a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:41.389095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cb41d-fc90-46de-bf2d-b75b2dddff27 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:41.396295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 84/171] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:41 2022] GET /identity/v3/users/6277667b84b54cbc9a6cc4d5288e746a => generated 289 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:41.396295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cb41d-fc90-46de-bf2d-b75b2dddff27 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:41.427886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5790cec-8b62-4fb8-b134-4234ddf08cff None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:41.448967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:41.470806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec694928-9ca7-486b-ace8-01538775f191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:41.472077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec694928-9ca7-486b-ace8-01538775f191 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.473093 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec694928-9ca7-486b-ace8-01538775f191 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.473525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec694928-9ca7-486b-ace8-01538775f191 None admin] PATH_INFO: `/v3/users/4bd8fa54875c432abf00243484dacb64` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.475079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec694928-9ca7-486b-ace8-01538775f191 None admin] RBAC: Authorizing `identity:get_user(user_id=4bd8fa54875c432abf00243484dacb64)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:41.477840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec694928-9ca7-486b-ace8-01538775f191 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:41.479608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 88/172] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:41 2022] GET /identity/v3/users/4bd8fa54875c432abf00243484dacb64 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:41.484088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5127d8-f2ce-4f61-a316-9aa66d1de730 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.484088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5127d8-f2ce-4f61-a316-9aa66d1de730 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.484088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5127d8-f2ce-4f61-a316-9aa66d1de730 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.484088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 89/173] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:41.484088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5127d8-f2ce-4f61-a316-9aa66d1de730 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:41.527654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cb41d-fc90-46de-bf2d-b75b2dddff27 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:41.572066 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752b1bac-6ed3-4694-86fe-e66cdff070e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:41.573273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b1bac-6ed3-4694-86fe-e66cdff070e9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.574394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b1bac-6ed3-4694-86fe-e66cdff070e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.574853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752b1bac-6ed3-4694-86fe-e66cdff070e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.596353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:41.620961 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5127d8-f2ce-4f61-a316-9aa66d1de730 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:41.662136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eada0d-e9bc-4671-905f-ed913eb27cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:41.663073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eada0d-e9bc-4671-905f-ed913eb27cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.663366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eada0d-e9bc-4671-905f-ed913eb27cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.663645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eada0d-e9bc-4671-905f-ed913eb27cc8 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.664333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b1bac-6ed3-4694-86fe-e66cdff070e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:41.664842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eada0d-e9bc-4671-905f-ed913eb27cc8 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:41.665876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752b1bac-6ed3-4694-86fe-e66cdff070e9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:41.666313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eada0d-e9bc-4671-905f-ed913eb27cc8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:41.668182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 90/174] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:41 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:41.668789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eada0d-e9bc-4671-905f-ed913eb27cc8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:41.720827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:41.775363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 85/175] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:41.777474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691f29c-45c5-4681-a0c4-76c932fb0db8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.777880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691f29c-45c5-4681-a0c4-76c932fb0db8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.778168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d691f29c-45c5-4681-a0c4-76c932fb0db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.784439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eada0d-e9bc-4671-905f-ed913eb27cc8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:41.794290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d691f29c-45c5-4681-a0c4-76c932fb0db8 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:41.795480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d691f29c-45c5-4681-a0c4-76c932fb0db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:41.821402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbd448-9143-4406-a865-2b89c95879bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:41.822057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbd448-9143-4406-a865-2b89c95879bf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.822346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbd448-9143-4406-a865-2b89c95879bf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.822626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcbd448-9143-4406-a865-2b89c95879bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.845234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:41.866739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 86/176] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:41.868300 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0c456-ff16-44c4-b559-e3a1ea8471d7 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:41.868634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0c456-ff16-44c4-b559-e3a1ea8471d7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:41.868911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d0c456-ff16-44c4-b559-e3a1ea8471d7 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:41.870355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 87/177] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:41.874158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0c456-ff16-44c4-b559-e3a1ea8471d7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:41.902290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbd448-9143-4406-a865-2b89c95879bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:41.903863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcbd448-9143-4406-a865-2b89c95879bf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:41.959138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:41.983686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d0c456-ff16-44c4-b559-e3a1ea8471d7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:42.008799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 91/178] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:42.009870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbd448-9143-4406-a865-2b89c95879bf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:42.026339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d399a-f704-4b8d-97c1-1d16ddf6a545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:42.027510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d399a-f704-4b8d-97c1-1d16ddf6a545 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.028039 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d399a-f704-4b8d-97c1-1d16ddf6a545 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.028489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d399a-f704-4b8d-97c1-1d16ddf6a545 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286/users/6277667b84b54cbc9a6cc4d5288e746a/roles/4f7a6e180d6d454bb3458ca27d7d822f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.030623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d399a-f704-4b8d-97c1-1d16ddf6a545 None admin] RBAC: Authorizing `identity:create_grant(project_id=23a2e92a012a4c6995bc96295899d286, user_id=6277667b84b54cbc9a6cc4d5288e746a, role_id=4f7a6e180d6d454bb3458ca27d7d822f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:42.032867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d399a-f704-4b8d-97c1-1d16ddf6a545 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:42.057408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 88/179] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:13:41 2022] PUT /identity/v3/projects/23a2e92a012a4c6995bc96295899d286/users/6277667b84b54cbc9a6cc4d5288e746a/roles/4f7a6e180d6d454bb3458ca27d7d822f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:13:42.057408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e433cd-6a6b-4f89-8aa6-427274b34789 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.057408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e433cd-6a6b-4f89-8aa6-427274b34789 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.057408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e433cd-6a6b-4f89-8aa6-427274b34789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.075464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:42.078095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-44e433cd-6a6b-4f89-8aa6-427274b34789 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:42.080984 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-44e433cd-6a6b-4f89-8aa6-427274b34789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:42.131485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcbd448-9143-4406-a865-2b89c95879bf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:42.170559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4949d-3ef4-4639-9978-bbb52bef5b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:42.171457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4949d-3ef4-4639-9978-bbb52bef5b43 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.171745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4949d-3ef4-4639-9978-bbb52bef5b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.172023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e4949d-3ef4-4639-9978-bbb52bef5b43 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.173917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4949d-3ef4-4639-9978-bbb52bef5b43 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:42.175410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e4949d-3ef4-4639-9978-bbb52bef5b43 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:42.181966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 92/180] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:42.181966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4949d-3ef4-4639-9978-bbb52bef5b43 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:42.182217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 89/181] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:42.183018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e433cd-6a6b-4f89-8aa6-427274b34789 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:42.264261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:42.302393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e433cd-6a6b-4f89-8aa6-427274b34789 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:42.307941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e4949d-3ef4-4639-9978-bbb52bef5b43 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:42.328578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:42.342961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487bcc9-440e-4c5f-9893-56dd67fde570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:42.343687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894d07e-215a-4533-95cc-e3185c60db49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:42.343933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487bcc9-440e-4c5f-9893-56dd67fde570 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.344315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487bcc9-440e-4c5f-9893-56dd67fde570 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.344663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894d07e-215a-4533-95cc-e3185c60db49 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.344999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894d07e-215a-4533-95cc-e3185c60db49 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.345280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3894d07e-215a-4533-95cc-e3185c60db49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.345399 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487bcc9-440e-4c5f-9893-56dd67fde570 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.346673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487bcc9-440e-4c5f-9893-56dd67fde570 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:42.348128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a487bcc9-440e-4c5f-9893-56dd67fde570 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:42.349886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 90/182] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:42.351181 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a4fe2-937a-418f-90ce-4d6d9e975f71 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.351469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a4fe2-937a-418f-90ce-4d6d9e975f71 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.351744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a4fe2-937a-418f-90ce-4d6d9e975f71 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.353148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 91/183] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:42.357550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a4fe2-937a-418f-90ce-4d6d9e975f71 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:42.363410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3894d07e-215a-4533-95cc-e3185c60db49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3894d07e-215a-4533-95cc-e3185c60db49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:42.373743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:42.374653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:42.376252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 93/184] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:42.377896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894d07e-215a-4533-95cc-e3185c60db49 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:42.473254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a4fe2-937a-418f-90ce-4d6d9e975f71 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:42.477907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3894d07e-215a-4533-95cc-e3185c60db49 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:42.509581 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b74d3-3eba-4658-991b-8d04089979dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:42.510241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b74d3-3eba-4658-991b-8d04089979dd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.510531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b74d3-3eba-4658-991b-8d04089979dd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.510815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946b74d3-3eba-4658-991b-8d04089979dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.511664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b74d3-3eba-4658-991b-8d04089979dd None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=23a2e92a012a4c6995bc96295899d286, user.id=417a441bd00d4df89cd837724e786c1e) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:42.511979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b74d3-3eba-4658-991b-8d04089979dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:42.513446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946b74d3-3eba-4658-991b-8d04089979dd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:42.515675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba547e66-0e8b-485d-b29b-3b9be83bb3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:42.516353 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:42.517088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547e66-0e8b-485d-b29b-3b9be83bb3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.517795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547e66-0e8b-485d-b29b-3b9be83bb3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.518103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba547e66-0e8b-485d-b29b-3b9be83bb3c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.521314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 92/185] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/role_assignments?user.id=417a441bd00d4df89cd837724e786c1e&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:42.522494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b74d3-3eba-4658-991b-8d04089979dd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:42.541703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba547e66-0e8b-485d-b29b-3b9be83bb3c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ba547e66-0e8b-485d-b29b-3b9be83bb3c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:42.554243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:42.555379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:42.556377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:42.561470 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 94/186] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:42.563804 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218972-7399-4eba-965d-cc35101ea545 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.564219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218972-7399-4eba-965d-cc35101ea545 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.564564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0218972-7399-4eba-965d-cc35101ea545 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.584508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e0218972-7399-4eba-965d-cc35101ea545 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:42.585300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e0218972-7399-4eba-965d-cc35101ea545 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:42.627963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:42.666048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946b74d3-3eba-4658-991b-8d04089979dd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:42.673253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 95/187] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:42.675075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218972-7399-4eba-965d-cc35101ea545 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:42.715461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16405023-7dec-46fa-861f-650ccb372e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:42.716363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16405023-7dec-46fa-861f-650ccb372e2b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.716807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16405023-7dec-46fa-861f-650ccb372e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.718576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16405023-7dec-46fa-861f-650ccb372e2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.719591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16405023-7dec-46fa-861f-650ccb372e2b None admin] RBAC: Adding query filter params (role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=23a2e92a012a4c6995bc96295899d286, user.id=4bd8fa54875c432abf00243484dacb64) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:42.720188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16405023-7dec-46fa-861f-650ccb372e2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:42.721824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16405023-7dec-46fa-861f-650ccb372e2b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:42.730648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 93/188] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/role_assignments?user.id=4bd8fa54875c432abf00243484dacb64&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=3b6f494c47a6407284d1d344940e34cd => generated 608 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:42.732250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16405023-7dec-46fa-861f-650ccb372e2b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:42.817943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:42.826089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0218972-7399-4eba-965d-cc35101ea545 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:42.876027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16405023-7dec-46fa-861f-650ccb372e2b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:42.876363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed322478-a8cd-4ad5-9dd0-7e82a94d8f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:42.877056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed322478-a8cd-4ad5-9dd0-7e82a94d8f81 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.877340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed322478-a8cd-4ad5-9dd0-7e82a94d8f81 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.877651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed322478-a8cd-4ad5-9dd0-7e82a94d8f81 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.878427 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed322478-a8cd-4ad5-9dd0-7e82a94d8f81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:42.878736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed322478-a8cd-4ad5-9dd0-7e82a94d8f81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:42.880161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed322478-a8cd-4ad5-9dd0-7e82a94d8f81 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:42.892262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 96/189] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:42.893549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed322478-a8cd-4ad5-9dd0-7e82a94d8f81 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:42.930655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de4090-297d-42a5-b4dd-82b60ef2595a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:42.931343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de4090-297d-42a5-b4dd-82b60ef2595a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:42.931805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de4090-297d-42a5-b4dd-82b60ef2595a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:42.932091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de4090-297d-42a5-b4dd-82b60ef2595a None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:42.933206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de4090-297d-42a5-b4dd-82b60ef2595a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:42.933549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de4090-297d-42a5-b4dd-82b60ef2595a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:42.934954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de4090-297d-42a5-b4dd-82b60ef2595a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:42.940774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:42.948318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 94/190] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:42.949158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de4090-297d-42a5-b4dd-82b60ef2595a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:43.027373 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed322478-a8cd-4ad5-9dd0-7e82a94d8f81 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:43.063670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e435e-7bf6-4e95-91d7-2be82de84d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:43.064780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e435e-7bf6-4e95-91d7-2be82de84d8d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:43.065063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e435e-7bf6-4e95-91d7-2be82de84d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:43.065336 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814e435e-7bf6-4e95-91d7-2be82de84d8d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:43.066552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e435e-7bf6-4e95-91d7-2be82de84d8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:43.068057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814e435e-7bf6-4e95-91d7-2be82de84d8d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:43.069875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 97/191] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:43.073108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de4090-297d-42a5-b4dd-82b60ef2595a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:43.074391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e435e-7bf6-4e95-91d7-2be82de84d8d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:43.100562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:43.111279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe5ec2-b934-4cc3-a4c0-0cae677bb679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:43.112480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe5ec2-b934-4cc3-a4c0-0cae677bb679 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:43.113034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe5ec2-b934-4cc3-a4c0-0cae677bb679 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:43.113534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe5ec2-b934-4cc3-a4c0-0cae677bb679 None admin] PATH_INFO: `/v3/users/02807162302e47e0997bf147a01fed1a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:43.127662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe5ec2-b934-4cc3-a4c0-0cae677bb679 None admin] RBAC: Authorizing `identity:get_user(user_id=02807162302e47e0997bf147a01fed1a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:43.129450 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fe5ec2-b934-4cc3-a4c0-0cae677bb679 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:43.131435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 95/192] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:42 2022] GET /identity/v3/users/02807162302e47e0997bf147a01fed1a => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:43.132430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe5ec2-b934-4cc3-a4c0-0cae677bb679 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:43.186943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:43.190603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814e435e-7bf6-4e95-91d7-2be82de84d8d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:43.228976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921db00a-3104-45ee-befb-b8b6db952811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:43.229932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921db00a-3104-45ee-befb-b8b6db952811 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:43.230221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921db00a-3104-45ee-befb-b8b6db952811 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:43.230545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921db00a-3104-45ee-befb-b8b6db952811 None admin] PATH_INFO: `/v3/users/02807162302e47e0997bf147a01fed1a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:43.231936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921db00a-3104-45ee-befb-b8b6db952811 None admin] RBAC: Authorizing `identity:get_user(user_id=02807162302e47e0997bf147a01fed1a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:43.233477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921db00a-3104-45ee-befb-b8b6db952811 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:43.240289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 98/193] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:43 2022] GET /identity/v3/users/02807162302e47e0997bf147a01fed1a => generated 283 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:43.240289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921db00a-3104-45ee-befb-b8b6db952811 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:43.251482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fe5ec2-b934-4cc3-a4c0-0cae677bb679 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:43.294071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69bd9f-8c1a-4f7b-9f34-26685865e5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:43.294991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69bd9f-8c1a-4f7b-9f34-26685865e5e5 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:43.295746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69bd9f-8c1a-4f7b-9f34-26685865e5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:43.296031 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f69bd9f-8c1a-4f7b-9f34-26685865e5e5 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:43.297022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69bd9f-8c1a-4f7b-9f34-26685865e5e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:43.299297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f69bd9f-8c1a-4f7b-9f34-26685865e5e5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:43.322538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 96/194] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:13:43 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:43.323584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69bd9f-8c1a-4f7b-9f34-26685865e5e5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:43.348760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921db00a-3104-45ee-befb-b8b6db952811 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:43.390348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:43.397443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa251-4391-4ce9-8485-221a2f915e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:43.399561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa251-4391-4ce9-8485-221a2f915e08 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:43.399767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa251-4391-4ce9-8485-221a2f915e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:43.400051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2aa251-4391-4ce9-8485-221a2f915e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:43.451521 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:43.471136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f69bd9f-8c1a-4f7b-9f34-26685865e5e5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:43.497774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa251-4391-4ce9-8485-221a2f915e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:43.499410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2aa251-4391-4ce9-8485-221a2f915e08 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:43.513007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd48975-3352-4aeb-a76e-9c73ab2022e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:43.513708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd48975-3352-4aeb-a76e-9c73ab2022e2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:43.513996 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd48975-3352-4aeb-a76e-9c73ab2022e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:43.514433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd48975-3352-4aeb-a76e-9c73ab2022e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:43.602510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd48975-3352-4aeb-a76e-9c73ab2022e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:43.603974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd48975-3352-4aeb-a76e-9c73ab2022e2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:43.627161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 99/195] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:43.636362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa251-4391-4ce9-8485-221a2f915e08 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:43.651842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:43.717793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:43.730621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 97/196] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:43.736881 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd48975-3352-4aeb-a76e-9c73ab2022e2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:43.762459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2aa251-4391-4ce9-8485-221a2f915e08 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:43.811245 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4eea3-ed34-4b3f-8e66-5c53f8c39984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:43.812393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4eea3-ed34-4b3f-8e66-5c53f8c39984 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:43.812684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4eea3-ed34-4b3f-8e66-5c53f8c39984 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:43.813513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4eea3-ed34-4b3f-8e66-5c53f8c39984 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:43.815064 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4eea3-ed34-4b3f-8e66-5c53f8c39984 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:43.817370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4eea3-ed34-4b3f-8e66-5c53f8c39984 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:43.819691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 100/197] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:43 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:43.826378 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4eea3-ed34-4b3f-8e66-5c53f8c39984 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:43.863129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd48975-3352-4aeb-a76e-9c73ab2022e2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:43.912460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e2d9f-e9b8-48ed-a2f7-d2602b8c368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:43.913727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e2d9f-e9b8-48ed-a2f7-d2602b8c368e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:43.914227 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e2d9f-e9b8-48ed-a2f7-d2602b8c368e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:43.914698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4e2d9f-e9b8-48ed-a2f7-d2602b8c368e None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:43.916104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e2d9f-e9b8-48ed-a2f7-d2602b8c368e None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:43.917799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4e2d9f-e9b8-48ed-a2f7-d2602b8c368e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:43.920076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 98/198] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:43 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:43.926965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e2d9f-e9b8-48ed-a2f7-d2602b8c368e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:43.940694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:43.948319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4eea3-ed34-4b3f-8e66-5c53f8c39984 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:43.983948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d6b4c5-a844-4c84-bdd0-82113c2d40f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:43.985557 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6b4c5-a844-4c84-bdd0-82113c2d40f0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:43.986266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6b4c5-a844-4c84-bdd0-82113c2d40f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:43.986718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d6b4c5-a844-4c84-bdd0-82113c2d40f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:43.987730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6b4c5-a844-4c84-bdd0-82113c2d40f0 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b, user.id=02807162302e47e0997bf147a01fed1a) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:43.988227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6b4c5-a844-4c84-bdd0-82113c2d40f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:43.989884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d6b4c5-a844-4c84-bdd0-82113c2d40f0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:43.993477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:44.003932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 101/199] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:43 2022] GET /identity/v3/role_assignments?user.id=02807162302e47e0997bf147a01fed1a&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:44.050108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4e2d9f-e9b8-48ed-a2f7-d2602b8c368e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:44.088821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b89079-3074-456d-9f88-2637d116b774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:44.089569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b89079-3074-456d-9f88-2637d116b774 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:44.089853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b89079-3074-456d-9f88-2637d116b774 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:44.090131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b89079-3074-456d-9f88-2637d116b774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:44.090980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b89079-3074-456d-9f88-2637d116b774 None admin] RBAC: Adding query filter params (role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b, user.id=02807162302e47e0997bf147a01fed1a) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:44.091296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b89079-3074-456d-9f88-2637d116b774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:44.092756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b89079-3074-456d-9f88-2637d116b774 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:44.099702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 99/200] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:43 2022] GET /identity/v3/role_assignments?user.id=02807162302e47e0997bf147a01fed1a&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=3b6f494c47a6407284d1d344940e34cd => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:44.259462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b2ed7-1f13-4d6e-9ff6-ffa0f19769d3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:44.260093 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b2ed7-1f13-4d6e-9ff6-ffa0f19769d3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:44.260590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b2ed7-1f13-4d6e-9ff6-ffa0f19769d3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:44.262225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 102/201] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:44.268291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7c96-57b1-48f3-befc-f3a30138865f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:44.268611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7c96-57b1-48f3-befc-f3a30138865f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:44.268859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1a7c96-57b1-48f3-befc-f3a30138865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:44.302068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ce1a7c96-57b1-48f3-befc-f3a30138865f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:44.303344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ce1a7c96-57b1-48f3-befc-f3a30138865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:44.385412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 100/202] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:44.409312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b2ed7-1f13-4d6e-9ff6-ffa0f19769d3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:44.520402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b2ed7-1f13-4d6e-9ff6-ffa0f19769d3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:44.557714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e00c8-441d-4d34-b158-c1b11479bc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:44.558861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e00c8-441d-4d34-b158-c1b11479bc69 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:44.559337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e00c8-441d-4d34-b158-c1b11479bc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:44.559800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767e00c8-441d-4d34-b158-c1b11479bc69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:44.571963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767e00c8-441d-4d34-b158-c1b11479bc69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-767e00c8-441d-4d34-b158-c1b11479bc69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:44.580764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:44.581628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:44.582433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:44.585073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 103/203] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:44.591052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7c96-57b1-48f3-befc-f3a30138865f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:44.600446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:44.717619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1a7c96-57b1-48f3-befc-f3a30138865f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:44.765504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b0ba5-8c1e-44fc-8c1c-c7c4ff06dfb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:44.767301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b0ba5-8c1e-44fc-8c1c-c7c4ff06dfb9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:44.767818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b0ba5-8c1e-44fc-8c1c-c7c4ff06dfb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:44.768294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75b0ba5-8c1e-44fc-8c1c-c7c4ff06dfb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:44.769257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b0ba5-8c1e-44fc-8c1c-c7c4ff06dfb9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:44.769775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b0ba5-8c1e-44fc-8c1c-c7c4ff06dfb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:44.771414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75b0ba5-8c1e-44fc-8c1c-c7c4ff06dfb9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:44.786886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 101/204] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:44.795768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e00c8-441d-4d34-b158-c1b11479bc69 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:44.864545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:44.911526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767e00c8-441d-4d34-b158-c1b11479bc69 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:44.951253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb545b48-1bcf-43c2-92b0-b7f98fca1075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:44.952491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb545b48-1bcf-43c2-92b0-b7f98fca1075 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:44.952972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb545b48-1bcf-43c2-92b0-b7f98fca1075 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:44.953470 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb545b48-1bcf-43c2-92b0-b7f98fca1075 None admin] PATH_INFO: `/v3/users/6277667b84b54cbc9a6cc4d5288e746a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:44.955145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb545b48-1bcf-43c2-92b0-b7f98fca1075 None admin] RBAC: Authorizing `identity:get_user(user_id=6277667b84b54cbc9a6cc4d5288e746a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:44.956830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb545b48-1bcf-43c2-92b0-b7f98fca1075 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:44.958992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 104/205] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:44 2022] GET /identity/v3/users/6277667b84b54cbc9a6cc4d5288e746a => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:44.964549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b0ba5-8c1e-44fc-8c1c-c7c4ff06dfb9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:45.036950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:45.090231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75b0ba5-8c1e-44fc-8c1c-c7c4ff06dfb9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:45.138605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac2bf9-a7f6-43d8-88bd-a36764ad0b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:45.139272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac2bf9-a7f6-43d8-88bd-a36764ad0b1a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:45.139562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac2bf9-a7f6-43d8-88bd-a36764ad0b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:45.139843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccac2bf9-a7f6-43d8-88bd-a36764ad0b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:45.232765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac2bf9-a7f6-43d8-88bd-a36764ad0b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:45.234259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccac2bf9-a7f6-43d8-88bd-a36764ad0b1a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:45.331325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:45.390216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 102/206] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:45.396173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb545b48-1bcf-43c2-92b0-b7f98fca1075 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:45.514540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb545b48-1bcf-43c2-92b0-b7f98fca1075 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:45.549797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:45.552169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016d20e-3f8f-48b9-b1d9-3caa4339ce9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:45.553276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016d20e-3f8f-48b9-b1d9-3caa4339ce9f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:45.553921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016d20e-3f8f-48b9-b1d9-3caa4339ce9f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:45.558908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016d20e-3f8f-48b9-b1d9-3caa4339ce9f None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:45.558908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016d20e-3f8f-48b9-b1d9-3caa4339ce9f None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:45.558908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016d20e-3f8f-48b9-b1d9-3caa4339ce9f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:45.565544 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 105/207] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:45 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:45.567341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac2bf9-a7f6-43d8-88bd-a36764ad0b1a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:45.683162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccac2bf9-a7f6-43d8-88bd-a36764ad0b1a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:45.730610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28743874-3f06-4343-ad40-282f74a50fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:45.731622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28743874-3f06-4343-ad40-282f74a50fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:45.732121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28743874-3f06-4343-ad40-282f74a50fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:45.732590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28743874-3f06-4343-ad40-282f74a50fc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:45.733653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28743874-3f06-4343-ad40-282f74a50fc7 None admin] RBAC: Adding query filter params (role.id=4f7a6e180d6d454bb3458ca27d7d822f, scope.project.id=23a2e92a012a4c6995bc96295899d286, user.id=6277667b84b54cbc9a6cc4d5288e746a) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:45.734160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28743874-3f06-4343-ad40-282f74a50fc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:45.736645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28743874-3f06-4343-ad40-282f74a50fc7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:45.743242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 103/208] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:45 2022] GET /identity/v3/role_assignments?user.id=6277667b84b54cbc9a6cc4d5288e746a&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=4f7a6e180d6d454bb3458ca27d7d822f => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:46.044667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdb672-8e43-45ae-b3ee-bed20d3a0169 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.045386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdb672-8e43-45ae-b3ee-bed20d3a0169 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.045889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cdb672-8e43-45ae-b3ee-bed20d3a0169 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.048505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 106/209] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:46.054023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f102b-cc4b-4a88-b934-31b81ab70645 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.054863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f102b-cc4b-4a88-b934-31b81ab70645 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.055352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f102b-cc4b-4a88-b934-31b81ab70645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.074456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-073f102b-cc4b-4a88-b934-31b81ab70645 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:46.075980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-073f102b-cc4b-4a88-b934-31b81ab70645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:46.101361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:46.148955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 104/210] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:46.175946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdb672-8e43-45ae-b3ee-bed20d3a0169 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:46.180721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9081c-d095-4bda-9ee6-1b9ec965a870 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.181016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9081c-d095-4bda-9ee6-1b9ec965a870 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.181295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b9081c-d095-4bda-9ee6-1b9ec965a870 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.182725 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 105/211] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:46.188591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce26e6-8cb8-4795-b6a3-7fbdb9dca692 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.189090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce26e6-8cb8-4795-b6a3-7fbdb9dca692 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.189564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce26e6-8cb8-4795-b6a3-7fbdb9dca692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.212251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-93ce26e6-8cb8-4795-b6a3-7fbdb9dca692 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:46.212251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-93ce26e6-8cb8-4795-b6a3-7fbdb9dca692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:46.300873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 106/212] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:46.306718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cdb672-8e43-45ae-b3ee-bed20d3a0169 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:46.325579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce26e6-8cb8-4795-b6a3-7fbdb9dca692 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:46.346892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12136b-a3e6-4f71-a9cd-c9e3e07a51c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:46.347697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12136b-a3e6-4f71-a9cd-c9e3e07a51c5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.347982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12136b-a3e6-4f71-a9cd-c9e3e07a51c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.348255 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12136b-a3e6-4f71-a9cd-c9e3e07a51c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.349606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12136b-a3e6-4f71-a9cd-c9e3e07a51c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:46.351050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e12136b-a3e6-4f71-a9cd-c9e3e07a51c5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:46.352883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 107/213] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:46.358296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12136b-a3e6-4f71-a9cd-c9e3e07a51c5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:46.426351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:46.431250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:46.446433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce26e6-8cb8-4795-b6a3-7fbdb9dca692 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:46.468895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12136b-a3e6-4f71-a9cd-c9e3e07a51c5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:46.483638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b92c7-8ce6-4adb-aac7-6bd9ca7c16ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:46.485313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b92c7-8ce6-4adb-aac7-6bd9ca7c16ba None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.486203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b92c7-8ce6-4adb-aac7-6bd9ca7c16ba None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.486761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b92c7-8ce6-4adb-aac7-6bd9ca7c16ba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.498641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591b92c7-8ce6-4adb-aac7-6bd9ca7c16ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:46.504147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cb4dea-01c5-4009-8f1c-f42e4d18bbb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:46.504872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb4dea-01c5-4009-8f1c-f42e4d18bbb2 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.505161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb4dea-01c5-4009-8f1c-f42e4d18bbb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.505479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cb4dea-01c5-4009-8f1c-f42e4d18bbb2 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.506214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb4dea-01c5-4009-8f1c-f42e4d18bbb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:46.507650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cb4dea-01c5-4009-8f1c-f42e4d18bbb2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-591b92c7-8ce6-4adb-aac7-6bd9ca7c16ba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:46.508857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:46.509775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:46.510754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:46.517995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 107/214] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:46.519302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b92c7-8ce6-4adb-aac7-6bd9ca7c16ba None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:46.538043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 108/215] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:13:46 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:46.594013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e27c3-0387-4902-b405-0723677147eb None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.594763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e27c3-0387-4902-b405-0723677147eb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.595046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e27c3-0387-4902-b405-0723677147eb None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.596895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 109/216] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:46.603226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b5ea-ffd8-48c8-abe1-4a8c66387922 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.603662 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b5ea-ffd8-48c8-abe1-4a8c66387922 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.604044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463b5ea-ffd8-48c8-abe1-4a8c66387922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.622709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2463b5ea-ffd8-48c8-abe1-4a8c66387922 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:46.624244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2463b5ea-ffd8-48c8-abe1-4a8c66387922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:46.643197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b92c7-8ce6-4adb-aac7-6bd9ca7c16ba None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:46.687193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ba505-0b60-41e4-86e1-daa8689d0a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:46.688413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ba505-0b60-41e4-86e1-daa8689d0a9a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.688703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ba505-0b60-41e4-86e1-daa8689d0a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.688977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135ba505-0b60-41e4-86e1-daa8689d0a9a None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.689788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ba505-0b60-41e4-86e1-daa8689d0a9a None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:46.690206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ba505-0b60-41e4-86e1-daa8689d0a9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:46.692076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135ba505-0b60-41e4-86e1-daa8689d0a9a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:46.702177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 108/217] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:46.702357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 110/218] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:46.710236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ba505-0b60-41e4-86e1-daa8689d0a9a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:46.725781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:46.726242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463b5ea-ffd8-48c8-abe1-4a8c66387922 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:46.789584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:46.837995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135ba505-0b60-41e4-86e1-daa8689d0a9a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:46.862283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463b5ea-ffd8-48c8-abe1-4a8c66387922 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:46.886553 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4e000-81f6-430f-8192-39d0908b280a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:46.888414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e000-81f6-430f-8192-39d0908b280a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.888696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e000-81f6-430f-8192-39d0908b280a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.888971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4e000-81f6-430f-8192-39d0908b280a None admin] PATH_INFO: `/v3/users/a7386f5742a54dd49a8a240d5836a2a0` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.909635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4e000-81f6-430f-8192-39d0908b280a None admin] RBAC: Authorizing `identity:get_user(user_id=a7386f5742a54dd49a8a240d5836a2a0)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:46.911220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4e000-81f6-430f-8192-39d0908b280a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:46.912300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9f5a6-17b6-4732-95e8-07142cb4f4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:46.913148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 109/219] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:46 2022] GET /identity/v3/users/a7386f5742a54dd49a8a240d5836a2a0 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:46.913253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f5a6-17b6-4732-95e8-07142cb4f4cb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:46.913523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f5a6-17b6-4732-95e8-07142cb4f4cb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:46.913710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9f5a6-17b6-4732-95e8-07142cb4f4cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:46.918417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4e000-81f6-430f-8192-39d0908b280a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:46.926864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9f5a6-17b6-4732-95e8-07142cb4f4cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b7c9f5a6-17b6-4732-95e8-07142cb4f4cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:46.934714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:46.935872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:46.936790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 111/220] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:13:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:13:46.943065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9f5a6-17b6-4732-95e8-07142cb4f4cb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:47.012743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:47.055664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4e000-81f6-430f-8192-39d0908b280a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:47.071895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9f5a6-17b6-4732-95e8-07142cb4f4cb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:47.085801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:47.096680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9d392b-d7e5-482e-bc5f-2ec0a73391f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:47.097809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d392b-d7e5-482e-bc5f-2ec0a73391f8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.098103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d392b-d7e5-482e-bc5f-2ec0a73391f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.098654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9d392b-d7e5-482e-bc5f-2ec0a73391f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.113610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee599766-5707-4d45-8e90-f52e1faeffc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:47.114242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee599766-5707-4d45-8e90-f52e1faeffc9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.114897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee599766-5707-4d45-8e90-f52e1faeffc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.115173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee599766-5707-4d45-8e90-f52e1faeffc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.116220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee599766-5707-4d45-8e90-f52e1faeffc9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:47.116529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee599766-5707-4d45-8e90-f52e1faeffc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:47.118703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee599766-5707-4d45-8e90-f52e1faeffc9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:47.131404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 112/221] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:47.132772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc382-6b63-4aa3-8ef5-d3fc010f1a59 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.133499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc382-6b63-4aa3-8ef5-d3fc010f1a59 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.133794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc382-6b63-4aa3-8ef5-d3fc010f1a59 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.135286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 113/222] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:47.136997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc382-6b63-4aa3-8ef5-d3fc010f1a59 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:47.192627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d392b-d7e5-482e-bc5f-2ec0a73391f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:47.194577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9d392b-d7e5-482e-bc5f-2ec0a73391f8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:47.270815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc382-6b63-4aa3-8ef5-d3fc010f1a59 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:47.279541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:47.316445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b52cb-76f7-4c86-a22b-173939fa4e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:47.317485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b52cb-76f7-4c86-a22b-173939fa4e59 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.317838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b52cb-76f7-4c86-a22b-173939fa4e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.318183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93b52cb-76f7-4c86-a22b-173939fa4e59 None admin] PATH_INFO: `/v3/users/02807162302e47e0997bf147a01fed1a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.319172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 110/223] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:13:47.320656 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519f4a9-9109-4f84-b70c-0772f3d751d6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.321105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519f4a9-9109-4f84-b70c-0772f3d751d6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.321562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519f4a9-9109-4f84-b70c-0772f3d751d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.323980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b52cb-76f7-4c86-a22b-173939fa4e59 None admin] RBAC: Authorizing `identity:get_user(user_id=02807162302e47e0997bf147a01fed1a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:47.325576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93b52cb-76f7-4c86-a22b-173939fa4e59 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:47.329343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 114/224] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:47 2022] GET /identity/v3/users/02807162302e47e0997bf147a01fed1a => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:47.330265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b52cb-76f7-4c86-a22b-173939fa4e59 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:47.340740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5519f4a9-9109-4f84-b70c-0772f3d751d6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:47.342367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5519f4a9-9109-4f84-b70c-0772f3d751d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:47.381674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:47.419924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 111/225] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:47.420588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519f4a9-9109-4f84-b70c-0772f3d751d6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:47.456802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93b52cb-76f7-4c86-a22b-173939fa4e59 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:47.505261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa1c9-f6c6-4606-ad65-6d35ac5c7a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:47.506269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa1c9-f6c6-4606-ad65-6d35ac5c7a67 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.506622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa1c9-f6c6-4606-ad65-6d35ac5c7a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.506964 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6aa1c9-f6c6-4606-ad65-6d35ac5c7a67 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.508245 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6aa1c9-f6c6-4606-ad65-6d35ac5c7a67 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:47.510360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6aa1c9-f6c6-4606-ad65-6d35ac5c7a67 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:47.513148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 115/226] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:47 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:47.517873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa1c9-f6c6-4606-ad65-6d35ac5c7a67 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:47.543134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:47.549653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519f4a9-9109-4f84-b70c-0772f3d751d6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:47.592585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5c4b7-a0ef-4825-961e-c4ea35842c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:47.593734 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5c4b7-a0ef-4825-961e-c4ea35842c07 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.594019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5c4b7-a0ef-4825-961e-c4ea35842c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.594827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd5c4b7-a0ef-4825-961e-c4ea35842c07 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.596923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5c4b7-a0ef-4825-961e-c4ea35842c07 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:47.598403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd5c4b7-a0ef-4825-961e-c4ea35842c07 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:47.600398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 112/227] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:47 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:47.601312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5c4b7-a0ef-4825-961e-c4ea35842c07 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:47.632868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6aa1c9-f6c6-4606-ad65-6d35ac5c7a67 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:47.644291 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:47.673122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f7eb4-704a-489c-a831-0e62ca48e1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:47.674202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7eb4-704a-489c-a831-0e62ca48e1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.674633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7eb4-704a-489c-a831-0e62ca48e1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.675170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f7eb4-704a-489c-a831-0e62ca48e1d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.686804 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670f7eb4-704a-489c-a831-0e62ca48e1d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-670f7eb4-704a-489c-a831-0e62ca48e1d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:47.696023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:47.697114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:47.697998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:47.700106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 116/228] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:13:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:47.700986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f7eb4-704a-489c-a831-0e62ca48e1d1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:47.717686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd5c4b7-a0ef-4825-961e-c4ea35842c07 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:47.760631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fe224-e11f-4d42-a27b-e30c9693c0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:47.761295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe224-e11f-4d42-a27b-e30c9693c0b1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.761605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe224-e11f-4d42-a27b-e30c9693c0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.761881 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623fe224-e11f-4d42-a27b-e30c9693c0b1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.762858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fe224-e11f-4d42-a27b-e30c9693c0b1 None admin] RBAC: Adding query filter params (role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b, user.id=a7386f5742a54dd49a8a240d5836a2a0) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:47.763182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fe224-e11f-4d42-a27b-e30c9693c0b1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:47.765200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623fe224-e11f-4d42-a27b-e30c9693c0b1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:47.777472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 113/229] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:47 2022] GET /identity/v3/role_assignments?user.id=a7386f5742a54dd49a8a240d5836a2a0&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:47.778852 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fe224-e11f-4d42-a27b-e30c9693c0b1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:47.837209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:47.870037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670f7eb4-704a-489c-a831-0e62ca48e1d1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:47.947900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc93a8-1595-478b-8bc7-9aef3b3121a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:47.949162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc93a8-1595-478b-8bc7-9aef3b3121a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:47.949705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc93a8-1595-478b-8bc7-9aef3b3121a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:47.950185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc93a8-1595-478b-8bc7-9aef3b3121a4 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/users/02807162302e47e0997bf147a01fed1a/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:47.952815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc93a8-1595-478b-8bc7-9aef3b3121a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=8faaa2ee2ac54d04b66c0dc66612d84b, user_id=02807162302e47e0997bf147a01fed1a, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:47.954774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc93a8-1595-478b-8bc7-9aef3b3121a4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:47.967597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623fe224-e11f-4d42-a27b-e30c9693c0b1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:47.980554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 117/230] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:13:47 2022] PUT /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/users/02807162302e47e0997bf147a01fed1a/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:13:48.026130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a20749-732f-4f2d-b22e-4cf0e30fe3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:48.026821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a20749-732f-4f2d-b22e-4cf0e30fe3cc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:48.027158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a20749-732f-4f2d-b22e-4cf0e30fe3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:48.027434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a20749-732f-4f2d-b22e-4cf0e30fe3cc None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:48.028259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a20749-732f-4f2d-b22e-4cf0e30fe3cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:48.028568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a20749-732f-4f2d-b22e-4cf0e30fe3cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:48.030191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a20749-732f-4f2d-b22e-4cf0e30fe3cc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:48.041094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 114/231] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:13:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:48.050584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc93a8-1595-478b-8bc7-9aef3b3121a4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:48.102346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:48.209343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc93a8-1595-478b-8bc7-9aef3b3121a4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:48.261202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ff3dc-c33c-4ae4-ac05-81ac8f16fd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:48.262114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ff3dc-c33c-4ae4-ac05-81ac8f16fd4d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:48.262407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ff3dc-c33c-4ae4-ac05-81ac8f16fd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:48.262688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ff3dc-c33c-4ae4-ac05-81ac8f16fd4d None admin] PATH_INFO: `/v3/users/02807162302e47e0997bf147a01fed1a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:48.266202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ff3dc-c33c-4ae4-ac05-81ac8f16fd4d None admin] RBAC: Authorizing `identity:get_user(user_id=02807162302e47e0997bf147a01fed1a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:48.267840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ff3dc-c33c-4ae4-ac05-81ac8f16fd4d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:48.269518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 118/232] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:48 2022] GET /identity/v3/users/02807162302e47e0997bf147a01fed1a => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:48.275057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a20749-732f-4f2d-b22e-4cf0e30fe3cc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:48.379559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:48.397420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a20749-732f-4f2d-b22e-4cf0e30fe3cc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:48.434160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d35c0dd-3f3d-4a0a-9b0c-79e4f9cb0489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:48.435084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35c0dd-3f3d-4a0a-9b0c-79e4f9cb0489 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:48.435378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35c0dd-3f3d-4a0a-9b0c-79e4f9cb0489 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:48.435920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35c0dd-3f3d-4a0a-9b0c-79e4f9cb0489 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:48.437124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35c0dd-3f3d-4a0a-9b0c-79e4f9cb0489 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:48.439525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d35c0dd-3f3d-4a0a-9b0c-79e4f9cb0489 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:48.441580 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 115/233] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:48 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:48.447282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ff3dc-c33c-4ae4-ac05-81ac8f16fd4d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:48.544439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:48.550920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ff3dc-c33c-4ae4-ac05-81ac8f16fd4d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:48.603465 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32991abe-238b-4bc6-bf31-50361f8b9f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:48.604379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991abe-238b-4bc6-bf31-50361f8b9f85 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:48.604665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991abe-238b-4bc6-bf31-50361f8b9f85 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:48.604944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32991abe-238b-4bc6-bf31-50361f8b9f85 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/users/02807162302e47e0997bf147a01fed1a/roles/3b6f494c47a6407284d1d344940e34cd` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:48.607380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32991abe-238b-4bc6-bf31-50361f8b9f85 None admin] RBAC: Authorizing `identity:create_grant(project_id=8faaa2ee2ac54d04b66c0dc66612d84b, user_id=02807162302e47e0997bf147a01fed1a, role_id=3b6f494c47a6407284d1d344940e34cd)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:48.608850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32991abe-238b-4bc6-bf31-50361f8b9f85 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:48.631269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 119/234] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:13:48 2022] PUT /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/users/02807162302e47e0997bf147a01fed1a/roles/3b6f494c47a6407284d1d344940e34cd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:13:49.105796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e898ae-c994-4e2e-89f9-0cc5b67f98b9 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:49.106171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e898ae-c994-4e2e-89f9-0cc5b67f98b9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:49.106359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e898ae-c994-4e2e-89f9-0cc5b67f98b9 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:49.108137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 116/235] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:49.113752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a935-1328-48b1-82d7-87ece78dad74 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:49.114196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a935-1328-48b1-82d7-87ece78dad74 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:49.114292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7894a935-1328-48b1-82d7-87ece78dad74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:49.137649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7894a935-1328-48b1-82d7-87ece78dad74 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:49.138874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7894a935-1328-48b1-82d7-87ece78dad74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:49.235701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 120/236] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:49.272002 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e898ae-c994-4e2e-89f9-0cc5b67f98b9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:49.292237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6cdcb-3def-4163-8a1f-eeb079ab5ed2 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:49.292526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6cdcb-3def-4163-8a1f-eeb079ab5ed2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:49.292807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6cdcb-3def-4163-8a1f-eeb079ab5ed2 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:49.294208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 121/237] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:49.299668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a596f-7174-4ee9-8ce1-b8949cec22c0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:49.299957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a596f-7174-4ee9-8ce1-b8949cec22c0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:49.300239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a596f-7174-4ee9-8ce1-b8949cec22c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:49.327828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2b8a596f-7174-4ee9-8ce1-b8949cec22c0 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:49.331573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2b8a596f-7174-4ee9-8ce1-b8949cec22c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:49.427260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e898ae-c994-4e2e-89f9-0cc5b67f98b9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:49.495515 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 122/238] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:49.499116 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27298c6-6415-468f-9b85-00083079548a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:49.499934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27298c6-6415-468f-9b85-00083079548a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:49.500218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27298c6-6415-468f-9b85-00083079548a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:49.501215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27298c6-6415-468f-9b85-00083079548a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:49.513921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27298c6-6415-468f-9b85-00083079548a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-e27298c6-6415-468f-9b85-00083079548a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:49.523360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:49.524421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:49.525226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 117/239] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:49.530670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a596f-7174-4ee9-8ce1-b8949cec22c0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:49.545161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27298c6-6415-468f-9b85-00083079548a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:49.575758 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:49.587704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:49.685431 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27298c6-6415-468f-9b85-00083079548a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:49.728073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08470436-62ca-4cdb-8105-36017a5b2c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:49.729974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08470436-62ca-4cdb-8105-36017a5b2c94 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:49.730484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08470436-62ca-4cdb-8105-36017a5b2c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:49.731103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08470436-62ca-4cdb-8105-36017a5b2c94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:49.733723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08470436-62ca-4cdb-8105-36017a5b2c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:49.736494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08470436-62ca-4cdb-8105-36017a5b2c94 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:49.739012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 118/240] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:49.744087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08470436-62ca-4cdb-8105-36017a5b2c94 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:49.764002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a596f-7174-4ee9-8ce1-b8949cec22c0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:49.817564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee7292-6429-48c8-8843-fbc46cb6a672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:49.817564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee7292-6429-48c8-8843-fbc46cb6a672 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:49.817564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee7292-6429-48c8-8843-fbc46cb6a672 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:49.817564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee7292-6429-48c8-8843-fbc46cb6a672 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:49.817973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee7292-6429-48c8-8843-fbc46cb6a672 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:49.818407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee7292-6429-48c8-8843-fbc46cb6a672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:49.821118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee7292-6429-48c8-8843-fbc46cb6a672 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:49.841133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 123/241] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:49 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:49.853117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee7292-6429-48c8-8843-fbc46cb6a672 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:49.916765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08470436-62ca-4cdb-8105-36017a5b2c94 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:49.925781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:49.956827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39028-96ad-463b-99d6-c1574e37d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:49.957875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39028-96ad-463b-99d6-c1574e37d57c None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:49.958348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39028-96ad-463b-99d6-c1574e37d57c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:49.958818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d39028-96ad-463b-99d6-c1574e37d57c None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:49.959729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39028-96ad-463b-99d6-c1574e37d57c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:49.961401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d39028-96ad-463b-99d6-c1574e37d57c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:49.996931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 119/242] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:13:49 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:13:50.036062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:50.068669 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee7292-6429-48c8-8843-fbc46cb6a672 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:50.121907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a234699-8871-42f5-ba49-34d1be72b2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:50.123657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a234699-8871-42f5-ba49-34d1be72b2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:50.124170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a234699-8871-42f5-ba49-34d1be72b2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:50.124662 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a234699-8871-42f5-ba49-34d1be72b2b8 None admin] PATH_INFO: `/v3/users/eae599dd98c948009537b36f8ca78024` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:50.155415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a234699-8871-42f5-ba49-34d1be72b2b8 None admin] RBAC: Authorizing `identity:get_user(user_id=eae599dd98c948009537b36f8ca78024)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:50.158867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a234699-8871-42f5-ba49-34d1be72b2b8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:50.162223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 124/243] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:49 2022] GET /identity/v3/users/eae599dd98c948009537b36f8ca78024 => generated 297 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:50.170665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39028-96ad-463b-99d6-c1574e37d57c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:50.255531 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979c3ed-d57a-4c32-8e14-c2d8cbf06fc0 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:50.256146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979c3ed-d57a-4c32-8e14-c2d8cbf06fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:50.256620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979c3ed-d57a-4c32-8e14-c2d8cbf06fc0 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:50.258432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 125/244] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:50.263927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef284666-b14c-4dd8-a41b-69be88e6990d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:50.264492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef284666-b14c-4dd8-a41b-69be88e6990d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:50.264967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef284666-b14c-4dd8-a41b-69be88e6990d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:50.285315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ef284666-b14c-4dd8-a41b-69be88e6990d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:50.286867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ef284666-b14c-4dd8-a41b-69be88e6990d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:50.311688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d39028-96ad-463b-99d6-c1574e37d57c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:50.366820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f31344b-bc64-489e-909b-b7d8a7710cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:50.367492 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31344b-bc64-489e-909b-b7d8a7710cab None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:50.367779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31344b-bc64-489e-909b-b7d8a7710cab None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:50.368057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f31344b-bc64-489e-909b-b7d8a7710cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:50.388381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 126/245] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:50.392473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:50.415030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef284666-b14c-4dd8-a41b-69be88e6990d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:50.467206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31344b-bc64-489e-909b-b7d8a7710cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:50.468177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f31344b-bc64-489e-909b-b7d8a7710cab None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:50.509299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:50.549198 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef284666-b14c-4dd8-a41b-69be88e6990d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:50.598411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 120/246] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:50.600080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18785835-9494-49ad-a75d-f7362b871a51 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:50.600551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18785835-9494-49ad-a75d-f7362b871a51 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:50.601006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18785835-9494-49ad-a75d-f7362b871a51 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:50.605400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89b39a-75b9-41b8-91f4-eedbd2cf6024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:50.605400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89b39a-75b9-41b8-91f4-eedbd2cf6024 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:50.605400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89b39a-75b9-41b8-91f4-eedbd2cf6024 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:50.605400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89b39a-75b9-41b8-91f4-eedbd2cf6024 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:50.606737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 121/247] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:50.607941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18785835-9494-49ad-a75d-f7362b871a51 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:50.622846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89b39a-75b9-41b8-91f4-eedbd2cf6024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-df89b39a-75b9-41b8-91f4-eedbd2cf6024 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:50.634842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:50.635836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:50.636624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 127/248] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:50.637281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341021a5-d906-4223-b6df-9b857863eb60 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:50.637860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341021a5-d906-4223-b6df-9b857863eb60 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:50.637955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341021a5-d906-4223-b6df-9b857863eb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:50.658578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-341021a5-d906-4223-b6df-9b857863eb60 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:50.659845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-341021a5-d906-4223-b6df-9b857863eb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:50.674933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:50.748717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18785835-9494-49ad-a75d-f7362b871a51 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:50.750144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 128/249] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:50.755251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341021a5-d906-4223-b6df-9b857863eb60 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:50.808162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f81a6d-69bf-4c89-b25f-7f80fa6c46a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:50.809002 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f81a6d-69bf-4c89-b25f-7f80fa6c46a3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:50.809289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f81a6d-69bf-4c89-b25f-7f80fa6c46a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:50.809585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f81a6d-69bf-4c89-b25f-7f80fa6c46a3 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:50.811439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f81a6d-69bf-4c89-b25f-7f80fa6c46a3 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:50.813699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f81a6d-69bf-4c89-b25f-7f80fa6c46a3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:50.815643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 122/250] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:50 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:50.816311 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f81a6d-69bf-4c89-b25f-7f80fa6c46a3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:50.835198 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:50.890234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341021a5-d906-4223-b6df-9b857863eb60 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:50.952974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f81a6d-69bf-4c89-b25f-7f80fa6c46a3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:50.964736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2989e-3e9a-4385-a0fd-b22c3cba185c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:50.965452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2989e-3e9a-4385-a0fd-b22c3cba185c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:50.965747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2989e-3e9a-4385-a0fd-b22c3cba185c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:50.966028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2989e-3e9a-4385-a0fd-b22c3cba185c None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:50.966852 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:50.967491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2989e-3e9a-4385-a0fd-b22c3cba185c None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:50.967885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2989e-3e9a-4385-a0fd-b22c3cba185c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:50.977587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e2989e-3e9a-4385-a0fd-b22c3cba185c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:50.995864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 129/251] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:50.997050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2989e-3e9a-4385-a0fd-b22c3cba185c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:51.003127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374a7b3-8c97-4bfc-97d9-f61454e11cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:51.003795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374a7b3-8c97-4bfc-97d9-f61454e11cdf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.004076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374a7b3-8c97-4bfc-97d9-f61454e11cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.004351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1374a7b3-8c97-4bfc-97d9-f61454e11cdf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.025051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1374a7b3-8c97-4bfc-97d9-f61454e11cdf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-1374a7b3-8c97-4bfc-97d9-f61454e11cdf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:51.043895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:51.044827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:51.045638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 123/252] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:13:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:51.046289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374a7b3-8c97-4bfc-97d9-f61454e11cdf None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:51.169906 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1374a7b3-8c97-4bfc-97d9-f61454e11cdf None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:51.205838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1118ab0-473e-4116-9166-c8116d1b7404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:51.206774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1118ab0-473e-4116-9166-c8116d1b7404 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.207058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1118ab0-473e-4116-9166-c8116d1b7404 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.207379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1118ab0-473e-4116-9166-c8116d1b7404 None admin] PATH_INFO: `/v3/users/a7386f5742a54dd49a8a240d5836a2a0` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.208578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1118ab0-473e-4116-9166-c8116d1b7404 None admin] RBAC: Authorizing `identity:get_user(user_id=a7386f5742a54dd49a8a240d5836a2a0)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:51.210046 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1118ab0-473e-4116-9166-c8116d1b7404 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:51.211974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 124/253] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:51 2022] GET /identity/v3/users/a7386f5742a54dd49a8a240d5836a2a0 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:51.212742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1118ab0-473e-4116-9166-c8116d1b7404 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:51.268861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2989e-3e9a-4385-a0fd-b22c3cba185c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:51.282272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:51.301783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:51.318005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439314d7-013a-463a-aec3-5255afe56af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:51.318986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439314d7-013a-463a-aec3-5255afe56af7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.319271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439314d7-013a-463a-aec3-5255afe56af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.319553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439314d7-013a-463a-aec3-5255afe56af7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.320627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439314d7-013a-463a-aec3-5255afe56af7 None admin] RBAC: Adding query filter params (role.id=4f7a6e180d6d454bb3458ca27d7d822f, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b, user.id=eae599dd98c948009537b36f8ca78024) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:51.320942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439314d7-013a-463a-aec3-5255afe56af7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:51.322823 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439314d7-013a-463a-aec3-5255afe56af7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:51.330172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 130/254] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:50 2022] GET /identity/v3/role_assignments?user.id=eae599dd98c948009537b36f8ca78024&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=4f7a6e180d6d454bb3458ca27d7d822f => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:51.331265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439314d7-013a-463a-aec3-5255afe56af7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:51.426436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1118ab0-473e-4116-9166-c8116d1b7404 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:51.488438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b770e7b-9d96-4fe5-8278-37a4ef22043f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:51.489244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b770e7b-9d96-4fe5-8278-37a4ef22043f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.489478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b770e7b-9d96-4fe5-8278-37a4ef22043f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.489805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b770e7b-9d96-4fe5-8278-37a4ef22043f None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.490752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b770e7b-9d96-4fe5-8278-37a4ef22043f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:51.490999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b770e7b-9d96-4fe5-8278-37a4ef22043f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:51.492400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b770e7b-9d96-4fe5-8278-37a4ef22043f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:51.503550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439314d7-013a-463a-aec3-5255afe56af7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:51.510493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 125/255] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:51.517072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b770e7b-9d96-4fe5-8278-37a4ef22043f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:51.557839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b19d49-2e77-429b-850b-344eef0d8d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:51.558722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b19d49-2e77-429b-850b-344eef0d8d47 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.559010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b19d49-2e77-429b-850b-344eef0d8d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.559290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b19d49-2e77-429b-850b-344eef0d8d47 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.562329 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b19d49-2e77-429b-850b-344eef0d8d47 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:51.564606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b19d49-2e77-429b-850b-344eef0d8d47 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:51.566731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 131/256] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:51 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:51.572660 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b19d49-2e77-429b-850b-344eef0d8d47 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:51.630400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b770e7b-9d96-4fe5-8278-37a4ef22043f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:51.632706 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:51.669390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:51.673281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180af7a-8ca2-4f15-8dab-3cac955c2b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:51.675139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180af7a-8ca2-4f15-8dab-3cac955c2b61 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.675423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180af7a-8ca2-4f15-8dab-3cac955c2b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.675699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180af7a-8ca2-4f15-8dab-3cac955c2b61 None admin] PATH_INFO: `/v3/users/02807162302e47e0997bf147a01fed1a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.677008 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180af7a-8ca2-4f15-8dab-3cac955c2b61 None admin] RBAC: Authorizing `identity:get_user(user_id=02807162302e47e0997bf147a01fed1a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:51.678721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180af7a-8ca2-4f15-8dab-3cac955c2b61 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:51.680532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 126/257] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:51 2022] GET /identity/v3/users/02807162302e47e0997bf147a01fed1a => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:51.681848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e222f8-65e7-4458-b8a7-6b80bb95f4d8 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.682132 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e222f8-65e7-4458-b8a7-6b80bb95f4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.682428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e222f8-65e7-4458-b8a7-6b80bb95f4d8 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.683647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 127/258] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:51.689859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e222f8-65e7-4458-b8a7-6b80bb95f4d8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:51.757611 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b19d49-2e77-429b-850b-344eef0d8d47 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:51.823644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30e7f05-2fac-4780-876e-94dd3491df27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:51.824537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e7f05-2fac-4780-876e-94dd3491df27 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.824822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e7f05-2fac-4780-876e-94dd3491df27 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.825096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30e7f05-2fac-4780-876e-94dd3491df27 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/users/a7386f5742a54dd49a8a240d5836a2a0/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.827389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e7f05-2fac-4780-876e-94dd3491df27 None admin] RBAC: Authorizing `identity:create_grant(project_id=8faaa2ee2ac54d04b66c0dc66612d84b, user_id=a7386f5742a54dd49a8a240d5836a2a0, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:51.829038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30e7f05-2fac-4780-876e-94dd3491df27 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:51.848381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 132/259] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:13:51 2022] PUT /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/users/a7386f5742a54dd49a8a240d5836a2a0/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:13:51.852380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5482fa-a72a-4dd7-801f-9c510961aff0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.852380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5482fa-a72a-4dd7-801f-9c510961aff0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.852380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5482fa-a72a-4dd7-801f-9c510961aff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.869599 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e222f8-65e7-4458-b8a7-6b80bb95f4d8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:51.869976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2b5482fa-a72a-4dd7-801f-9c510961aff0 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:51.871322 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2b5482fa-a72a-4dd7-801f-9c510961aff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:51.947535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad2463-e4b0-4d87-9660-ba8641b791ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:51.948550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad2463-e4b0-4d87-9660-ba8641b791ab None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:51.949035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad2463-e4b0-4d87-9660-ba8641b791ab None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:51.949562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad2463-e4b0-4d87-9660-ba8641b791ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:51.959632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 133/260] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:51.991148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5482fa-a72a-4dd7-801f-9c510961aff0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:52.054653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:52.081201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:52.130682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad2463-e4b0-4d87-9660-ba8641b791ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:52.133056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ad2463-e4b0-4d87-9660-ba8641b791ab None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:52.184151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5482fa-a72a-4dd7-801f-9c510961aff0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:52.250118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890bb6f-a1d2-4ca3-8d8d-8eb66e718869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:52.251739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890bb6f-a1d2-4ca3-8d8d-8eb66e718869 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:52.252123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890bb6f-a1d2-4ca3-8d8d-8eb66e718869 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:52.252521 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890bb6f-a1d2-4ca3-8d8d-8eb66e718869 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:52.263423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890bb6f-a1d2-4ca3-8d8d-8eb66e718869 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1890bb6f-a1d2-4ca3-8d8d-8eb66e718869 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:52.272187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:52.273545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:52.274331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 134/261] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:13:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:52.283755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890bb6f-a1d2-4ca3-8d8d-8eb66e718869 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:52.319381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 128/262] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:13:52.325173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad2463-e4b0-4d87-9660-ba8641b791ab None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:52.493001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:52.553892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad2463-e4b0-4d87-9660-ba8641b791ab None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:52.558762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:52.562410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890bb6f-a1d2-4ca3-8d8d-8eb66e718869 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:52.637175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94048b3e-5b46-4cd4-a9f7-48833262a1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:52.637893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94048b3e-5b46-4cd4-a9f7-48833262a1ac None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:52.638183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94048b3e-5b46-4cd4-a9f7-48833262a1ac None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:52.638572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94048b3e-5b46-4cd4-a9f7-48833262a1ac None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:52.639027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bc51b-63e2-401b-9133-d80c902cd537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:52.639339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94048b3e-5b46-4cd4-a9f7-48833262a1ac None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:52.639653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94048b3e-5b46-4cd4-a9f7-48833262a1ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:52.639903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bc51b-63e2-401b-9133-d80c902cd537 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:52.640198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bc51b-63e2-401b-9133-d80c902cd537 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:52.640481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752bc51b-63e2-401b-9133-d80c902cd537 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:52.641806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94048b3e-5b46-4cd4-a9f7-48833262a1ac None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:52.646154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bc51b-63e2-401b-9133-d80c902cd537 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:52.647616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752bc51b-63e2-401b-9133-d80c902cd537 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:52.649709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 129/263] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:52 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:52.654962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bc51b-63e2-401b-9133-d80c902cd537 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:52.660557 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 135/264] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:13:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:52.674520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94048b3e-5b46-4cd4-a9f7-48833262a1ac None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:52.904706 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752bc51b-63e2-401b-9133-d80c902cd537 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:52.931521 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94048b3e-5b46-4cd4-a9f7-48833262a1ac None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:52.979980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcee01f-0cbe-4d6c-b985-f19ab9f731aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:52.980650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcee01f-0cbe-4d6c-b985-f19ab9f731aa None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:52.980942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcee01f-0cbe-4d6c-b985-f19ab9f731aa None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:52.981222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcee01f-0cbe-4d6c-b985-f19ab9f731aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:52.982117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcee01f-0cbe-4d6c-b985-f19ab9f731aa None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b, user.id=02807162302e47e0997bf147a01fed1a) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:52.982434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcee01f-0cbe-4d6c-b985-f19ab9f731aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:52.983890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcee01f-0cbe-4d6c-b985-f19ab9f731aa None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:52.998365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 130/265] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:52 2022] GET /identity/v3/role_assignments?user.id=02807162302e47e0997bf147a01fed1a&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 608 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:53.006134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8411685-2140-44fc-81d8-d8f14eab8fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:53.007039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8411685-2140-44fc-81d8-d8f14eab8fd0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.007335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8411685-2140-44fc-81d8-d8f14eab8fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.007610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8411685-2140-44fc-81d8-d8f14eab8fd0 None admin] PATH_INFO: `/v3/users/02807162302e47e0997bf147a01fed1a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.008879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8411685-2140-44fc-81d8-d8f14eab8fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=02807162302e47e0997bf147a01fed1a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:53.010372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8411685-2140-44fc-81d8-d8f14eab8fd0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:53.012021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 136/266] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:52 2022] GET /identity/v3/users/02807162302e47e0997bf147a01fed1a => generated 283 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:53.018166 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcee01f-0cbe-4d6c-b985-f19ab9f731aa None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:53.024340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:53.182451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcee01f-0cbe-4d6c-b985-f19ab9f731aa None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:53.223997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6edf55-f074-4cef-9fbf-078986ce074e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:53.229788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6edf55-f074-4cef-9fbf-078986ce074e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.230089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6edf55-f074-4cef-9fbf-078986ce074e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.230382 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6edf55-f074-4cef-9fbf-078986ce074e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.319294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6edf55-f074-4cef-9fbf-078986ce074e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:53.320786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6edf55-f074-4cef-9fbf-078986ce074e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:53.322886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:53.368454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c41ec-367b-4d6a-81ac-c1861473e43d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.368974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c41ec-367b-4d6a-81ac-c1861473e43d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.369242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c41ec-367b-4d6a-81ac-c1861473e43d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.370987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 137/267] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:53.376734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549af9d-f1d1-4cd8-9b14-3472fa3b5920 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.377086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549af9d-f1d1-4cd8-9b14-3472fa3b5920 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.377467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549af9d-f1d1-4cd8-9b14-3472fa3b5920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.397725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7549af9d-f1d1-4cd8-9b14-3472fa3b5920 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:53.399060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7549af9d-f1d1-4cd8-9b14-3472fa3b5920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:53.439493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:53.454069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 131/268] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:53.459347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6edf55-f074-4cef-9fbf-078986ce074e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:53.504979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 138/269] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:53.527773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ec75-1986-4ad3-979e-34014f930ae2 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.528101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ec75-1986-4ad3-979e-34014f930ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.528378 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2ec75-1986-4ad3-979e-34014f930ae2 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.529998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 139/270] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:53.534854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ec75-1986-4ad3-979e-34014f930ae2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:53.581949 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6edf55-f074-4cef-9fbf-078986ce074e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:53.583441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:53.628048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e83372-1c1d-4408-bc09-cac936565c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:53.629673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e83372-1c1d-4408-bc09-cac936565c6b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.630847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e83372-1c1d-4408-bc09-cac936565c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.631637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e83372-1c1d-4408-bc09-cac936565c6b None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.633325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e83372-1c1d-4408-bc09-cac936565c6b None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:53.635898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e83372-1c1d-4408-bc09-cac936565c6b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:53.638220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 132/271] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:53 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:53.640221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52ad28-4371-43c9-b299-7641c461d48a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.643375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52ad28-4371-43c9-b299-7641c461d48a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.643492 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c52ad28-4371-43c9-b299-7641c461d48a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.664069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1c52ad28-4371-43c9-b299-7641c461d48a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:53.665269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1c52ad28-4371-43c9-b299-7641c461d48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:53.677202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2ec75-1986-4ad3-979e-34014f930ae2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:53.725816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1611b1d-7238-4acb-93d0-0654e349cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:53.727133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611b1d-7238-4acb-93d0-0654e349cbcb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.727592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611b1d-7238-4acb-93d0-0654e349cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.728260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1611b1d-7238-4acb-93d0-0654e349cbcb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.730366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1611b1d-7238-4acb-93d0-0654e349cbcb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:53.732612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1611b1d-7238-4acb-93d0-0654e349cbcb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:53.735188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 140/272] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:53.736192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1611b1d-7238-4acb-93d0-0654e349cbcb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:53.745416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 133/273] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:53.745416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52ad28-4371-43c9-b299-7641c461d48a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:53.780601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:53.861043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1611b1d-7238-4acb-93d0-0654e349cbcb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:53.894707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:53.899229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1edfd-b6e9-49dc-a6b3-ebddbd2173cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:53.899610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c52ad28-4371-43c9-b299-7641c461d48a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:53.900251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1edfd-b6e9-49dc-a6b3-ebddbd2173cf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.900544 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1edfd-b6e9-49dc-a6b3-ebddbd2173cf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.900822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1edfd-b6e9-49dc-a6b3-ebddbd2173cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.901700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1edfd-b6e9-49dc-a6b3-ebddbd2173cf None admin] RBAC: Adding query filter params (role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b, user.id=02807162302e47e0997bf147a01fed1a) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:53.902191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1edfd-b6e9-49dc-a6b3-ebddbd2173cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:53.903811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d1edfd-b6e9-49dc-a6b3-ebddbd2173cf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:53.911349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 141/274] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:53 2022] GET /identity/v3/role_assignments?user.id=02807162302e47e0997bf147a01fed1a&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=3b6f494c47a6407284d1d344940e34cd => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:53.912202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1edfd-b6e9-49dc-a6b3-ebddbd2173cf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:53.950532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c8ca3-8a92-44ad-a817-372a56def2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:53.951373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c8ca3-8a92-44ad-a817-372a56def2e1 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:53.951761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c8ca3-8a92-44ad-a817-372a56def2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:53.952142 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c8ca3-8a92-44ad-a817-372a56def2e1 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:53.953329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c8ca3-8a92-44ad-a817-372a56def2e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:53.956014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c8ca3-8a92-44ad-a817-372a56def2e1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:54.002217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 134/275] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:13:53 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:54.029405 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:54.041719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1edfd-b6e9-49dc-a6b3-ebddbd2173cf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:54.083541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026bb90-bfb6-483b-9f18-f838114023f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:54.084645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026bb90-bfb6-483b-9f18-f838114023f9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.085078 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026bb90-bfb6-483b-9f18-f838114023f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.085538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026bb90-bfb6-483b-9f18-f838114023f9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.098615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4026bb90-bfb6-483b-9f18-f838114023f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4026bb90-bfb6-483b-9f18-f838114023f9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:54.107231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:54.108400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:54.109489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:54.112408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 142/276] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:54.122583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c8ca3-8a92-44ad-a817-372a56def2e1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:54.237708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1f8a1-7757-4cd3-be4e-16ac9d45a420 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.238001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1f8a1-7757-4cd3-be4e-16ac9d45a420 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.238343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1f8a1-7757-4cd3-be4e-16ac9d45a420 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.240145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c8ca3-8a92-44ad-a817-372a56def2e1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:54.241152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 143/277] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:54.247360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3583-a791-4b05-a3c9-401524e439f4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.247656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3583-a791-4b05-a3c9-401524e439f4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.248286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bd3583-a791-4b05-a3c9-401524e439f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.272066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d5bd3583-a791-4b05-a3c9-401524e439f4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:54.273198 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d5bd3583-a791-4b05-a3c9-401524e439f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:54.279382 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dd184-6922-45c1-9ce7-641c76f2bbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:54.280344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dd184-6922-45c1-9ce7-641c76f2bbcf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.280869 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dd184-6922-45c1-9ce7-641c76f2bbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.281342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56dd184-6922-45c1-9ce7-641c76f2bbcf None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.283595 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dd184-6922-45c1-9ce7-641c76f2bbcf None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:54.284125 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dd184-6922-45c1-9ce7-641c76f2bbcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:54.286240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56dd184-6922-45c1-9ce7-641c76f2bbcf None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:54.298134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 135/278] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:54.307033 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dd184-6922-45c1-9ce7-641c76f2bbcf None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:54.335657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:54.376364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 144/279] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:54.402285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd3583-a791-4b05-a3c9-401524e439f4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:54.443405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56dd184-6922-45c1-9ce7-641c76f2bbcf None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:54.492182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c0101-f5c1-418b-a537-9153568646cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:54.493292 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0101-f5c1-418b-a537-9153568646cd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.493615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0101-f5c1-418b-a537-9153568646cd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.494318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55c0101-f5c1-418b-a537-9153568646cd None admin] PATH_INFO: `/v3/users/0395ca349ac44c8ab0e948741cb34e14` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.509995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0101-f5c1-418b-a537-9153568646cd None admin] RBAC: Authorizing `identity:get_user(user_id=0395ca349ac44c8ab0e948741cb34e14)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:54.511511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55c0101-f5c1-418b-a537-9153568646cd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:54.513562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 136/280] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:54 2022] GET /identity/v3/users/0395ca349ac44c8ab0e948741cb34e14 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:54.518715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:54.520333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c0101-f5c1-418b-a537-9153568646cd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:54.545319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bd3583-a791-4b05-a3c9-401524e439f4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:54.594612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1776d909-8166-4df2-9ae8-8cb5ff1919e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:54.595766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776d909-8166-4df2-9ae8-8cb5ff1919e3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.596347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776d909-8166-4df2-9ae8-8cb5ff1919e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.596857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1776d909-8166-4df2-9ae8-8cb5ff1919e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.610876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1776d909-8166-4df2-9ae8-8cb5ff1919e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:54.616277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1776d909-8166-4df2-9ae8-8cb5ff1919e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:54.621994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:54.622874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:54.623646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 145/281] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:13:54.624970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848af1a-194f-41bb-995b-627fcd8e7277 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.625592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848af1a-194f-41bb-995b-627fcd8e7277 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.625847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4848af1a-194f-41bb-995b-627fcd8e7277 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.628528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 146/282] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:54.629490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848af1a-194f-41bb-995b-627fcd8e7277 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:54.655100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55c0101-f5c1-418b-a537-9153568646cd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:54.700571 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2724fe02-9d81-4471-b1ce-397412513de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:54.701284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724fe02-9d81-4471-b1ce-397412513de7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.701603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724fe02-9d81-4471-b1ce-397412513de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.701885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2724fe02-9d81-4471-b1ce-397412513de7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.703206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724fe02-9d81-4471-b1ce-397412513de7 None admin] RBAC: Adding query filter params (role.id=5a1c0efbb0f14248bf3a2f523478eef2, user.id=0395ca349ac44c8ab0e948741cb34e14, scope.system=all) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:54.705100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724fe02-9d81-4471-b1ce-397412513de7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:54.706891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2724fe02-9d81-4471-b1ce-397412513de7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:54.717473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 137/283] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jul 12 13:13:54 2022] GET /identity/v3/role_assignments?user.id=0395ca349ac44c8ab0e948741cb34e14&scope.system=all&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:54.719659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7a69e-d388-4c2d-a755-8a88811577e0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.720128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7a69e-d388-4c2d-a755-8a88811577e0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.720561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e7a69e-d388-4c2d-a755-8a88811577e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.760467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-54e7a69e-d388-4c2d-a755-8a88811577e0 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:54.761755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-54e7a69e-d388-4c2d-a755-8a88811577e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:54.772348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4848af1a-194f-41bb-995b-627fcd8e7277 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:54.838311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d695d-a6a6-40ee-91aa-6bc07a76a54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:54.838932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d695d-a6a6-40ee-91aa-6bc07a76a54b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:54.839216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d695d-a6a6-40ee-91aa-6bc07a76a54b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:54.839489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d695d-a6a6-40ee-91aa-6bc07a76a54b None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:54.840263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d695d-a6a6-40ee-91aa-6bc07a76a54b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:54.840570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d695d-a6a6-40ee-91aa-6bc07a76a54b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:54.842307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d695d-a6a6-40ee-91aa-6bc07a76a54b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:54.856371 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 147/284] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:54.862519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d695d-a6a6-40ee-91aa-6bc07a76a54b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:54.873756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 138/285] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:54.874636 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:54.900664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7a69e-d388-4c2d-a755-8a88811577e0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:54.992127 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d695d-a6a6-40ee-91aa-6bc07a76a54b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:55.001592 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:55.028356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611af4f1-ea13-4c9d-bbc9-0e1949c1bed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:55.029265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611af4f1-ea13-4c9d-bbc9-0e1949c1bed4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:55.029804 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611af4f1-ea13-4c9d-bbc9-0e1949c1bed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:55.029933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611af4f1-ea13-4c9d-bbc9-0e1949c1bed4 None admin] PATH_INFO: `/v3/users/eae599dd98c948009537b36f8ca78024` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:55.031120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611af4f1-ea13-4c9d-bbc9-0e1949c1bed4 None admin] RBAC: Authorizing `identity:get_user(user_id=eae599dd98c948009537b36f8ca78024)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:55.032945 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611af4f1-ea13-4c9d-bbc9-0e1949c1bed4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:55.039284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 148/286] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:54 2022] GET /identity/v3/users/eae599dd98c948009537b36f8ca78024 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:55.040475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611af4f1-ea13-4c9d-bbc9-0e1949c1bed4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:55.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e7a69e-d388-4c2d-a755-8a88811577e0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:55.089587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b4c98-4fd2-4bfe-afde-47a74e7b680a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:55.090806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b4c98-4fd2-4bfe-afde-47a74e7b680a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:55.091292 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b4c98-4fd2-4bfe-afde-47a74e7b680a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:55.091812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544b4c98-4fd2-4bfe-afde-47a74e7b680a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:55.103663 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544b4c98-4fd2-4bfe-afde-47a74e7b680a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-544b4c98-4fd2-4bfe-afde-47a74e7b680a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:55.112664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:55.113578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:55.114373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:55.117129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 139/287] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:13:55.130056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b4c98-4fd2-4bfe-afde-47a74e7b680a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:55.171000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:55.225486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611af4f1-ea13-4c9d-bbc9-0e1949c1bed4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:55.285639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55127ff-03ea-4f4b-8236-45321cd43795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:55.286511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55127ff-03ea-4f4b-8236-45321cd43795 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:55.286798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55127ff-03ea-4f4b-8236-45321cd43795 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:55.287078 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55127ff-03ea-4f4b-8236-45321cd43795 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:55.297548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55127ff-03ea-4f4b-8236-45321cd43795 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:55.297548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55127ff-03ea-4f4b-8236-45321cd43795 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:55.297779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 149/288] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:55 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:55.299597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55127ff-03ea-4f4b-8236-45321cd43795 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:55.335495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544b4c98-4fd2-4bfe-afde-47a74e7b680a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:55.409817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:55.415838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad6f4c-d147-4b73-be8c-4ba644036135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:55.416518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad6f4c-d147-4b73-be8c-4ba644036135 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:55.416809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad6f4c-d147-4b73-be8c-4ba644036135 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:55.417091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ad6f4c-d147-4b73-be8c-4ba644036135 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:55.417889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad6f4c-d147-4b73-be8c-4ba644036135 None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:55.418200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad6f4c-d147-4b73-be8c-4ba644036135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:55.418785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55127ff-03ea-4f4b-8236-45321cd43795 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:55.419601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ad6f4c-d147-4b73-be8c-4ba644036135 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:55.430875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 140/289] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:55.437119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad6f4c-d147-4b73-be8c-4ba644036135 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:55.483092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d6cc7-5b84-4c98-be2d-71a28871ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:55.485238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6cc7-5b84-4c98-be2d-71a28871ac64 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:55.485823 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6cc7-5b84-4c98-be2d-71a28871ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:55.487022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d6cc7-5b84-4c98-be2d-71a28871ac64 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/users/eae599dd98c948009537b36f8ca78024/roles/4f7a6e180d6d454bb3458ca27d7d822f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:55.490950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d6cc7-5b84-4c98-be2d-71a28871ac64 None admin] RBAC: Authorizing `identity:create_grant(project_id=8faaa2ee2ac54d04b66c0dc66612d84b, user_id=eae599dd98c948009537b36f8ca78024, role_id=4f7a6e180d6d454bb3458ca27d7d822f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:55.493280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d6cc7-5b84-4c98-be2d-71a28871ac64 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:55.517808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:55.522130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 150/290] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:13:55 2022] PUT /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/users/eae599dd98c948009537b36f8ca78024/roles/4f7a6e180d6d454bb3458ca27d7d822f => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:13:55.598652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ad6f4c-d147-4b73-be8c-4ba644036135 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:55.646770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2b32b-d0a6-4e55-98aa-a2206f5bb834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:55.648379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b32b-d0a6-4e55-98aa-a2206f5bb834 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:55.648820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b32b-d0a6-4e55-98aa-a2206f5bb834 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:55.649244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d2b32b-d0a6-4e55-98aa-a2206f5bb834 None admin] PATH_INFO: `/v3/users/a7386f5742a54dd49a8a240d5836a2a0` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:55.654318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2b32b-d0a6-4e55-98aa-a2206f5bb834 None admin] RBAC: Authorizing `identity:get_user(user_id=a7386f5742a54dd49a8a240d5836a2a0)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:55.656052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d2b32b-d0a6-4e55-98aa-a2206f5bb834 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:55.658250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 141/291] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:55 2022] GET /identity/v3/users/a7386f5742a54dd49a8a240d5836a2a0 => generated 297 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:55.664126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d6cc7-5b84-4c98-be2d-71a28871ac64 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:55.791835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d6cc7-5b84-4c98-be2d-71a28871ac64 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:55.833525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25677d60-55c7-4838-a4ac-6d3b653d0e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:55.834425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25677d60-55c7-4838-a4ac-6d3b653d0e35 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:55.834884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25677d60-55c7-4838-a4ac-6d3b653d0e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:55.835310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25677d60-55c7-4838-a4ac-6d3b653d0e35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:55.921607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:55.924657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25677d60-55c7-4838-a4ac-6d3b653d0e35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:55.926228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25677d60-55c7-4838-a4ac-6d3b653d0e35 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:56.035069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 151/292] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:56.036459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2b32b-d0a6-4e55-98aa-a2206f5bb834 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:56.108132 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:56.151582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d2b32b-d0a6-4e55-98aa-a2206f5bb834 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:56.186161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6305d89-970f-4756-b644-973b2d3a0698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:56.186944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305d89-970f-4756-b644-973b2d3a0698 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:56.187226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305d89-970f-4756-b644-973b2d3a0698 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:56.187498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6305d89-970f-4756-b644-973b2d3a0698 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:56.189301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6305d89-970f-4756-b644-973b2d3a0698 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:56.190780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6305d89-970f-4756-b644-973b2d3a0698 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:56.192778 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 142/293] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:56 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:56.198593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25677d60-55c7-4838-a4ac-6d3b653d0e35 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:56.361761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:56.365933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25677d60-55c7-4838-a4ac-6d3b653d0e35 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:56.411921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c0b77-a012-4c03-adcf-780879020687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:56.412590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c0b77-a012-4c03-adcf-780879020687 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:56.412879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c0b77-a012-4c03-adcf-780879020687 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:56.413158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c0b77-a012-4c03-adcf-780879020687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:56.414020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c0b77-a012-4c03-adcf-780879020687 None admin] RBAC: Adding query filter params (role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b, user.id=a7386f5742a54dd49a8a240d5836a2a0) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:56.414335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c0b77-a012-4c03-adcf-780879020687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:56.415785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c0b77-a012-4c03-adcf-780879020687 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:56.429940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 152/294] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:56 2022] GET /identity/v3/role_assignments?user.id=a7386f5742a54dd49a8a240d5836a2a0&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:56.640451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fdbf2-ed2c-41a4-bbdd-c7c0d3dd1831 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:56.640896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fdbf2-ed2c-41a4-bbdd-c7c0d3dd1831 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:56.641189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266fdbf2-ed2c-41a4-bbdd-c7c0d3dd1831 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:56.642721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 143/295] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:56.647993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b983f-c9d3-4c96-bd1d-56be75b22a57 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:56.648287 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b983f-c9d3-4c96-bd1d-56be75b22a57 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:56.648573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03b983f-c9d3-4c96-bd1d-56be75b22a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:56.674004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b03b983f-c9d3-4c96-bd1d-56be75b22a57 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:56.675409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b03b983f-c9d3-4c96-bd1d-56be75b22a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:56.751788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 153/296] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:56.774065 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fdbf2-ed2c-41a4-bbdd-c7c0d3dd1831 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:56.900829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266fdbf2-ed2c-41a4-bbdd-c7c0d3dd1831 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:56.929776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:56.940950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08902f36-8269-4741-a009-82539f329d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:56.941782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08902f36-8269-4741-a009-82539f329d5f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:56.942065 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08902f36-8269-4741-a009-82539f329d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:56.942337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08902f36-8269-4741-a009-82539f329d5f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:56.943567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08902f36-8269-4741-a009-82539f329d5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:56.944994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08902f36-8269-4741-a009-82539f329d5f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:56.946669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 144/297] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:56.952195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b983f-c9d3-4c96-bd1d-56be75b22a57 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:57.061969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:57.069533 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03b983f-c9d3-4c96-bd1d-56be75b22a57 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:57.104189 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cccbf31-0aa5-46a2-9c4b-8d26e2579e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:57.104863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccbf31-0aa5-46a2-9c4b-8d26e2579e7c None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.105145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccbf31-0aa5-46a2-9c4b-8d26e2579e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.105441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cccbf31-0aa5-46a2-9c4b-8d26e2579e7c None admin] PATH_INFO: `/v3/groups` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.106137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccbf31-0aa5-46a2-9c4b-8d26e2579e7c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:57.107627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cccbf31-0aa5-46a2-9c4b-8d26e2579e7c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:57.128858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 154/298] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Jul 12 13:13:56 2022] POST /identity/v3/groups => generated 229 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:13:57.143155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5458b4-4ac8-47d3-9763-fa8891743ad3 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.143441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5458b4-4ac8-47d3-9763-fa8891743ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.143914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5458b4-4ac8-47d3-9763-fa8891743ad3 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.145350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 145/299] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:57.151538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a4d9-2d10-4c64-8616-c31ff6f5a0b2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.151808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a4d9-2d10-4c64-8616-c31ff6f5a0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.152024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191a4d9-2d10-4c64-8616-c31ff6f5a0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.175768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3191a4d9-2d10-4c64-8616-c31ff6f5a0b2 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:57.177086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3191a4d9-2d10-4c64-8616-c31ff6f5a0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:57.199550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0391813-796d-4b0f-a99d-4a181bcb403f None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.200159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0391813-796d-4b0f-a99d-4a181bcb403f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.200661 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0391813-796d-4b0f-a99d-4a181bcb403f None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.202496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 146/300] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:13:57.211116 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c8068-e74f-4fd3-9c29-6a0bd88b9583 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.211858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c8068-e74f-4fd3-9c29-6a0bd88b9583 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.212348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052c8068-e74f-4fd3-9c29-6a0bd88b9583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.234378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-052c8068-e74f-4fd3-9c29-6a0bd88b9583 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:57.236164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-052c8068-e74f-4fd3-9c29-6a0bd88b9583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:57.260747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 155/301] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:13:57.289101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191a4d9-2d10-4c64-8616-c31ff6f5a0b2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:57.320303 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 147/302] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:57.343115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c8068-e74f-4fd3-9c29-6a0bd88b9583 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:57.391956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:57.412682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191a4d9-2d10-4c64-8616-c31ff6f5a0b2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:57.458118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09511b8f-4343-4b0c-84e1-8e10e1eab63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:57.459916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511b8f-4343-4b0c-84e1-8e10e1eab63c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.460548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511b8f-4343-4b0c-84e1-8e10e1eab63c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.461040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09511b8f-4343-4b0c-84e1-8e10e1eab63c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.474639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052c8068-e74f-4fd3-9c29-6a0bd88b9583 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:57.476772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09511b8f-4343-4b0c-84e1-8e10e1eab63c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-09511b8f-4343-4b0c-84e1-8e10e1eab63c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:57.487845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:57.488747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:57.489563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:13:57.495436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 156/303] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:13:57.496514 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09511b8f-4343-4b0c-84e1-8e10e1eab63c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:57.516696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:57.528516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a227e6-ddf4-44e2-b36c-e22235aaaba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:57.530190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a227e6-ddf4-44e2-b36c-e22235aaaba4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.530628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a227e6-ddf4-44e2-b36c-e22235aaaba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.531078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a227e6-ddf4-44e2-b36c-e22235aaaba4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.544985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a227e6-ddf4-44e2-b36c-e22235aaaba4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-e6a227e6-ddf4-44e2-b36c-e22235aaaba4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:57.557027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:57.558175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:13:57.559171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:57.567751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 148/304] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:13:57.568448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a227e6-ddf4-44e2-b36c-e22235aaaba4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:57.617170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09511b8f-4343-4b0c-84e1-8e10e1eab63c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:57.663577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9648c07-39a6-4a45-bf70-8cc067c55202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:57.664652 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9648c07-39a6-4a45-bf70-8cc067c55202 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.664999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9648c07-39a6-4a45-bf70-8cc067c55202 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.665281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9648c07-39a6-4a45-bf70-8cc067c55202 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.666354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9648c07-39a6-4a45-bf70-8cc067c55202 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:57.666688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9648c07-39a6-4a45-bf70-8cc067c55202 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:57.668393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9648c07-39a6-4a45-bf70-8cc067c55202 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:57.680452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 157/305] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:57.686012 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9648c07-39a6-4a45-bf70-8cc067c55202 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:57.691416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a227e6-ddf4-44e2-b36c-e22235aaaba4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:57.696830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:57.726293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af2c93-0931-4e6e-a26f-98c1f9fa5c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:57.726964 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af2c93-0931-4e6e-a26f-98c1f9fa5c42 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.727251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af2c93-0931-4e6e-a26f-98c1f9fa5c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.727524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96af2c93-0931-4e6e-a26f-98c1f9fa5c42 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.728287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af2c93-0931-4e6e-a26f-98c1f9fa5c42 None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:57.728589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af2c93-0931-4e6e-a26f-98c1f9fa5c42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:57.729993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96af2c93-0931-4e6e-a26f-98c1f9fa5c42 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:57.739289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 149/306] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:57.744441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af2c93-0931-4e6e-a26f-98c1f9fa5c42 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:57.794334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:57.800787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9648c07-39a6-4a45-bf70-8cc067c55202 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:57.838630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a18564-2917-49a1-8aa2-7c623a72d326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:57.839819 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a18564-2917-49a1-8aa2-7c623a72d326 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.840339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a18564-2917-49a1-8aa2-7c623a72d326 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.840800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a18564-2917-49a1-8aa2-7c623a72d326 None admin] PATH_INFO: `/v3/users/218b36f90a094ce29641a1acd8e07998` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.847475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96af2c93-0931-4e6e-a26f-98c1f9fa5c42 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:57.856422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a18564-2917-49a1-8aa2-7c623a72d326 None admin] RBAC: Authorizing `identity:get_user(user_id=218b36f90a094ce29641a1acd8e07998)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:57.858283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a18564-2917-49a1-8aa2-7c623a72d326 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:57.860666 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 158/307] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:57 2022] GET /identity/v3/users/218b36f90a094ce29641a1acd8e07998 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:57.866038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a18564-2917-49a1-8aa2-7c623a72d326 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:57.888087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43e615-e899-426c-ba65-beccddce58f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:57.888974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43e615-e899-426c-ba65-beccddce58f5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:57.889454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43e615-e899-426c-ba65-beccddce58f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:57.889838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a43e615-e899-426c-ba65-beccddce58f5 None admin] PATH_INFO: `/v3/users/0395ca349ac44c8ab0e948741cb34e14` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:57.891750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43e615-e899-426c-ba65-beccddce58f5 None admin] RBAC: Authorizing `identity:get_user(user_id=0395ca349ac44c8ab0e948741cb34e14)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:57.893358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a43e615-e899-426c-ba65-beccddce58f5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:57.895223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 150/308] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:57 2022] GET /identity/v3/users/0395ca349ac44c8ab0e948741cb34e14 => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:13:57.900651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43e615-e899-426c-ba65-beccddce58f5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:57.938708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:57.981191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a18564-2917-49a1-8aa2-7c623a72d326 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:58.013093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a43e615-e899-426c-ba65-beccddce58f5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:58.016127 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c0aea8-a974-440a-b337-4a93d8578776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:58.016995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0aea8-a974-440a-b337-4a93d8578776 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:58.017475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0aea8-a974-440a-b337-4a93d8578776 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:58.017892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c0aea8-a974-440a-b337-4a93d8578776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:58.018870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0aea8-a974-440a-b337-4a93d8578776 None admin] RBAC: Adding query filter params (role.id=4f7a6e180d6d454bb3458ca27d7d822f, user.id=218b36f90a094ce29641a1acd8e07998, scope.system=all) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:58.019340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0aea8-a974-440a-b337-4a93d8578776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:58.020910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c0aea8-a974-440a-b337-4a93d8578776 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:58.026739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 159/309] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jul 12 13:13:57 2022] GET /identity/v3/role_assignments?user.id=218b36f90a094ce29641a1acd8e07998&scope.system=all&role.id=4f7a6e180d6d454bb3458ca27d7d822f => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:58.032029 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:58.037862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24591f89-8f56-4175-82ac-ab2aaba70b6c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:58.038063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24591f89-8f56-4175-82ac-ab2aaba70b6c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:58.038442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24591f89-8f56-4175-82ac-ab2aaba70b6c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:58.040218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 160/310] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:58.046729 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff3b2d-1a7d-4a49-9add-3c75a3feb9c3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:58.047941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff3b2d-1a7d-4a49-9add-3c75a3feb9c3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:58.048055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ff3b2d-1a7d-4a49-9add-3c75a3feb9c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:58.056613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f23b2d-1946-4522-a340-ce76c57f7638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:58.057831 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f23b2d-1946-4522-a340-ce76c57f7638 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:58.058258 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f23b2d-1946-4522-a340-ce76c57f7638 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:58.058678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f23b2d-1946-4522-a340-ce76c57f7638 None admin] PATH_INFO: `/v3/system/users/0395ca349ac44c8ab0e948741cb34e14/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:58.060799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f23b2d-1946-4522-a340-ce76c57f7638 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0395ca349ac44c8ab0e948741cb34e14, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:58.062418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f23b2d-1946-4522-a340-ce76c57f7638 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:58.066324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b2ff3b2d-1a7d-4a49-9add-3c75a3feb9c3 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:58.068120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b2ff3b2d-1a7d-4a49-9add-3c75a3feb9c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:58.077493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 151/311] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Jul 12 13:13:57 2022] PUT /identity/v3/system/users/0395ca349ac44c8ab0e948741cb34e14/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:13:58.166436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 161/312] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:58.193189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f23b2d-1946-4522-a340-ce76c57f7638 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:58.316955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f23b2d-1946-4522-a340-ce76c57f7638 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:58.354149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f534f4-eed1-4ef5-8a78-8542c525db6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:58.354957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f534f4-eed1-4ef5-8a78-8542c525db6f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:58.355241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f534f4-eed1-4ef5-8a78-8542c525db6f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:58.355520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f534f4-eed1-4ef5-8a78-8542c525db6f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:58.366852 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f534f4-eed1-4ef5-8a78-8542c525db6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d7f534f4-eed1-4ef5-8a78-8542c525db6f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:13:58.374874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:13:58.376401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:13:58.377191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:13:58.381644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 152/313] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:13:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:13:58.383175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff3b2d-1a7d-4a49-9add-3c75a3feb9c3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:58.444396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:58.487099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ff3b2d-1a7d-4a49-9add-3c75a3feb9c3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:58.526708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abde79-fb82-421b-9a59-52e978f696d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:58.527720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abde79-fb82-421b-9a59-52e978f696d0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:58.528225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abde79-fb82-421b-9a59-52e978f696d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:58.528695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abde79-fb82-421b-9a59-52e978f696d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:58.529693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abde79-fb82-421b-9a59-52e978f696d0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:58.530193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abde79-fb82-421b-9a59-52e978f696d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:58.532335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abde79-fb82-421b-9a59-52e978f696d0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:58.541751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 162/314] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:13:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:58.557402 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f534f4-eed1-4ef5-8a78-8542c525db6f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:58.622047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:58.659209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f534f4-eed1-4ef5-8a78-8542c525db6f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:58.694731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f1115-122a-4237-821b-9b0437659bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:58.695824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f1115-122a-4237-821b-9b0437659bba None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:58.696267 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f1115-122a-4237-821b-9b0437659bba None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:58.696683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f1115-122a-4237-821b-9b0437659bba None admin] PATH_INFO: `/v3/users/eae599dd98c948009537b36f8ca78024` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:58.698051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f1115-122a-4237-821b-9b0437659bba None admin] RBAC: Authorizing `identity:get_user(user_id=eae599dd98c948009537b36f8ca78024)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:58.699619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f1115-122a-4237-821b-9b0437659bba None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:58.701772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 153/315] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:13:58 2022] GET /identity/v3/users/eae599dd98c948009537b36f8ca78024 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:58.709981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abde79-fb82-421b-9a59-52e978f696d0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:58.812712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abde79-fb82-421b-9a59-52e978f696d0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:58.837464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:58.852513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf59552-a941-45a3-9d95-4793a1afc280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:58.853406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf59552-a941-45a3-9d95-4793a1afc280 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:58.854892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf59552-a941-45a3-9d95-4793a1afc280 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:58.854892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf59552-a941-45a3-9d95-4793a1afc280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:58.926219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf59552-a941-45a3-9d95-4793a1afc280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:58.927930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf59552-a941-45a3-9d95-4793a1afc280 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:59.029416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 163/316] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:13:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:13:59.040147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f1115-122a-4237-821b-9b0437659bba None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:59.147167 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f1115-122a-4237-821b-9b0437659bba None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:59.156267 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:59.182286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c570ca4-cf78-4772-818d-78aa02e2bf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:59.183163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c570ca4-cf78-4772-818d-78aa02e2bf75 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:59.183447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c570ca4-cf78-4772-818d-78aa02e2bf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:59.183722 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c570ca4-cf78-4772-818d-78aa02e2bf75 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:59.184915 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c570ca4-cf78-4772-818d-78aa02e2bf75 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:59.186638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c570ca4-cf78-4772-818d-78aa02e2bf75 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:59.188678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 154/317] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:13:59 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:59.194547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf59552-a941-45a3-9d95-4793a1afc280 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:59.216437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:59.266761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c1756-ab99-43ee-9310-7c7bbf9db97c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:59.267415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c1756-ab99-43ee-9310-7c7bbf9db97c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:59.267892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04c1756-ab99-43ee-9310-7c7bbf9db97c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:59.269841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 155/318] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:13:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:13:59.274914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bc659-fadc-4f69-bc1d-f91d8706d73e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:59.275207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bc659-fadc-4f69-bc1d-f91d8706d73e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:59.275721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7bc659-fadc-4f69-bc1d-f91d8706d73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:59.293305 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-eb7bc659-fadc-4f69-bc1d-f91d8706d73e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:13:59.294910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-eb7bc659-fadc-4f69-bc1d-f91d8706d73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:13:59.322932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf59552-a941-45a3-9d95-4793a1afc280 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:59.367975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08537739-d02e-4241-9857-6b417e04dc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:59.368901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08537739-d02e-4241-9857-6b417e04dc49 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:59.369327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08537739-d02e-4241-9857-6b417e04dc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:59.369793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08537739-d02e-4241-9857-6b417e04dc49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:59.369992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 156/319] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:13:59.371101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08537739-d02e-4241-9857-6b417e04dc49 None admin] RBAC: Adding query filter params (role.id=4f7a6e180d6d454bb3458ca27d7d822f, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b, user.id=eae599dd98c948009537b36f8ca78024) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:13:59.371414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08537739-d02e-4241-9857-6b417e04dc49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:59.372850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08537739-d02e-4241-9857-6b417e04dc49 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:59.379875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 164/320] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:13:59 2022] GET /identity/v3/role_assignments?user.id=eae599dd98c948009537b36f8ca78024&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=4f7a6e180d6d454bb3458ca27d7d822f => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:59.394629 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bc659-fadc-4f69-bc1d-f91d8706d73e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:59.509624 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7bc659-fadc-4f69-bc1d-f91d8706d73e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:59.518613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:59.550530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c406da2-b4b4-4bf2-bfc9-4ff53ee3deec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:59.551247 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406da2-b4b4-4bf2-bfc9-4ff53ee3deec None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:59.551531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406da2-b4b4-4bf2-bfc9-4ff53ee3deec None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:59.551805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c406da2-b4b4-4bf2-bfc9-4ff53ee3deec None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:59.553679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c406da2-b4b4-4bf2-bfc9-4ff53ee3deec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:59.555120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c406da2-b4b4-4bf2-bfc9-4ff53ee3deec None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:59.556920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 157/321] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:13:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:13:59.563148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08537739-d02e-4241-9857-6b417e04dc49 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:13:59.661048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:13:59.678568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08537739-d02e-4241-9857-6b417e04dc49 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:13:59.713626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa468423-f19c-4ed9-9eda-0a6fc57f1f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:13:59.714611 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa468423-f19c-4ed9-9eda-0a6fc57f1f46 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:13:59.715120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa468423-f19c-4ed9-9eda-0a6fc57f1f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:13:59.715592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa468423-f19c-4ed9-9eda-0a6fc57f1f46 None admin] PATH_INFO: `/v3/groups` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:13:59.716502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa468423-f19c-4ed9-9eda-0a6fc57f1f46 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:13:59.718156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa468423-f19c-4ed9-9eda-0a6fc57f1f46 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:13:59.741501 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 165/322] 10.222.0.23 () {60 vars in 1134 bytes} [Tue Jul 12 13:13:59 2022] POST /identity/v3/groups => generated 226 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:00.261337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de23491-30f4-457d-946c-ca035978644f None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:00.261855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de23491-30f4-457d-946c-ca035978644f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:00.261930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de23491-30f4-457d-946c-ca035978644f None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:00.263482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 158/323] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:00.268685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4846-f74b-4cfd-96c8-b489c3040980 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:00.268978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4846-f74b-4cfd-96c8-b489c3040980 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:00.269261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4846-f74b-4cfd-96c8-b489c3040980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:00.287586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-23ee4846-f74b-4cfd-96c8-b489c3040980 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:00.288830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-23ee4846-f74b-4cfd-96c8-b489c3040980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:00.303565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a66bd6-e38f-4dff-a56c-e5cb10b84e02 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:00.304200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a66bd6-e38f-4dff-a56c-e5cb10b84e02 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:00.304682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a66bd6-e38f-4dff-a56c-e5cb10b84e02 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:00.306528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 159/324] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:00.311945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525a9a9-f6f7-4a50-a35e-c60170bc9ae9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:00.312477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525a9a9-f6f7-4a50-a35e-c60170bc9ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:00.313556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9525a9a9-f6f7-4a50-a35e-c60170bc9ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:00.333456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9525a9a9-f6f7-4a50-a35e-c60170bc9ae9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:00.335010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9525a9a9-f6f7-4a50-a35e-c60170bc9ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:00.398520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 166/325] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:00.425495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 160/326] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:00.428876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4846-f74b-4cfd-96c8-b489c3040980 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:00.454617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525a9a9-f6f7-4a50-a35e-c60170bc9ae9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:00.534490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:00.570885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee4846-f74b-4cfd-96c8-b489c3040980 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:00.601455 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9525a9a9-f6f7-4a50-a35e-c60170bc9ae9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:00.608137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:00.616426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802298ef-d6b5-4f66-a403-0d06f9e26803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:00.617246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802298ef-d6b5-4f66-a403-0d06f9e26803 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:00.617561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802298ef-d6b5-4f66-a403-0d06f9e26803 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:00.617839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802298ef-d6b5-4f66-a403-0d06f9e26803 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:00.631163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802298ef-d6b5-4f66-a403-0d06f9e26803 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-802298ef-d6b5-4f66-a403-0d06f9e26803 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:00.641524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:00.642687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:14:00.643486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:00.647889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 167/327] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:00.657568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802298ef-d6b5-4f66-a403-0d06f9e26803 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:00.687270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb081e38-0730-435b-b415-9d238001f1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:00.688140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081e38-0730-435b-b415-9d238001f1ce None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:00.688420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081e38-0730-435b-b415-9d238001f1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:00.688779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb081e38-0730-435b-b415-9d238001f1ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:00.716745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb081e38-0730-435b-b415-9d238001f1ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-cb081e38-0730-435b-b415-9d238001f1ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:00.725847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:00.726784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:00.727572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:00.733879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 161/328] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:00.739359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb081e38-0730-435b-b415-9d238001f1ce None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:00.777287 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802298ef-d6b5-4f66-a403-0d06f9e26803 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:00.825088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c05754-79c9-4ce3-9d9f-5ffa3c0dbb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:00.825718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c05754-79c9-4ce3-9d9f-5ffa3c0dbb96 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:00.826001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c05754-79c9-4ce3-9d9f-5ffa3c0dbb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:00.826280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c05754-79c9-4ce3-9d9f-5ffa3c0dbb96 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:00.827039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c05754-79c9-4ce3-9d9f-5ffa3c0dbb96 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:00.827345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c05754-79c9-4ce3-9d9f-5ffa3c0dbb96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:00.828880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c05754-79c9-4ce3-9d9f-5ffa3c0dbb96 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:00.833680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:00.848564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 168/329] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:14:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:00.854199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c05754-79c9-4ce3-9d9f-5ffa3c0dbb96 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:00.895736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb081e38-0730-435b-b415-9d238001f1ce None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:00.936934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6253aa-2a06-4b3e-bfd0-5909a985ae8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:00.938583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6253aa-2a06-4b3e-bfd0-5909a985ae8f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:00.939283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6253aa-2a06-4b3e-bfd0-5909a985ae8f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:00.939842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6253aa-2a06-4b3e-bfd0-5909a985ae8f None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:00.940966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6253aa-2a06-4b3e-bfd0-5909a985ae8f None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:00.941464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6253aa-2a06-4b3e-bfd0-5909a985ae8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:00.943306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6253aa-2a06-4b3e-bfd0-5909a985ae8f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:00.953989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 162/330] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:00.959114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6253aa-2a06-4b3e-bfd0-5909a985ae8f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:00.990224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:01.070761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6253aa-2a06-4b3e-bfd0-5909a985ae8f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:01.088713 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c05754-79c9-4ce3-9d9f-5ffa3c0dbb96 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:01.108108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45fe06-361e-4e33-acc2-b6c8bea08fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:01.109228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45fe06-361e-4e33-acc2-b6c8bea08fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:01.110432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45fe06-361e-4e33-acc2-b6c8bea08fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:01.110951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d45fe06-361e-4e33-acc2-b6c8bea08fc4 None admin] PATH_INFO: `/v3/users/0395ca349ac44c8ab0e948741cb34e14` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:01.112409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45fe06-361e-4e33-acc2-b6c8bea08fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=0395ca349ac44c8ab0e948741cb34e14)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:01.114037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d45fe06-361e-4e33-acc2-b6c8bea08fc4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:01.115820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 163/331] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:14:00 2022] GET /identity/v3/users/0395ca349ac44c8ab0e948741cb34e14 => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:01.121498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45fe06-361e-4e33-acc2-b6c8bea08fc4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:01.169790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288ad1c-e14f-4548-b036-1065d66fd8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:01.171042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288ad1c-e14f-4548-b036-1065d66fd8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:01.171499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288ad1c-e14f-4548-b036-1065d66fd8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:01.172562 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c288ad1c-e14f-4548-b036-1065d66fd8a5 None admin] PATH_INFO: `/v3/users/218b36f90a094ce29641a1acd8e07998` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:01.174661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288ad1c-e14f-4548-b036-1065d66fd8a5 None admin] RBAC: Authorizing `identity:get_user(user_id=218b36f90a094ce29641a1acd8e07998)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:01.176356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c288ad1c-e14f-4548-b036-1065d66fd8a5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:01.180164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 169/332] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:14:00 2022] GET /identity/v3/users/218b36f90a094ce29641a1acd8e07998 => generated 293 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:01.206509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288ad1c-e14f-4548-b036-1065d66fd8a5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:01.350389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:01.350877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:01.368775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d45fe06-361e-4e33-acc2-b6c8bea08fc4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:01.420879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c288ad1c-e14f-4548-b036-1065d66fd8a5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:01.475493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2879a2-b3fc-4f03-a8ba-33830694ebe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:01.476516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2879a2-b3fc-4f03-a8ba-33830694ebe8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:01.476964 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2879a2-b3fc-4f03-a8ba-33830694ebe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:01.477075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67562625-9031-4282-84f3-962dbfa99db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:01.477521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2879a2-b3fc-4f03-a8ba-33830694ebe8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:01.477979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67562625-9031-4282-84f3-962dbfa99db2 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:01.478423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67562625-9031-4282-84f3-962dbfa99db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:01.478534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2879a2-b3fc-4f03-a8ba-33830694ebe8 None admin] RBAC: Adding query filter params (role.id=5a1c0efbb0f14248bf3a2f523478eef2, user.id=0395ca349ac44c8ab0e948741cb34e14, scope.system=all) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:01.478821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2879a2-b3fc-4f03-a8ba-33830694ebe8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:01.479109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67562625-9031-4282-84f3-962dbfa99db2 None admin] PATH_INFO: `/v3/system/users/218b36f90a094ce29641a1acd8e07998/roles/4f7a6e180d6d454bb3458ca27d7d822f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:01.485403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2879a2-b3fc-4f03-a8ba-33830694ebe8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:01.486522 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67562625-9031-4282-84f3-962dbfa99db2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=218b36f90a094ce29641a1acd8e07998, role_id=4f7a6e180d6d454bb3458ca27d7d822f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:01.488106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67562625-9031-4282-84f3-962dbfa99db2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:01.503393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 170/333] 10.222.0.23 () {58 vars in 1334 bytes} [Tue Jul 12 13:14:01 2022] PUT /identity/v3/system/users/218b36f90a094ce29641a1acd8e07998/roles/4f7a6e180d6d454bb3458ca27d7d822f => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:01.504477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 164/334] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jul 12 13:14:01 2022] GET /identity/v3/role_assignments?user.id=0395ca349ac44c8ab0e948741cb34e14&scope.system=all&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 510 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:02.530047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259a475-b7c9-4db5-858e-f006aca99f40 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:02.530520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259a475-b7c9-4db5-858e-f006aca99f40 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:02.530520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a259a475-b7c9-4db5-858e-f006aca99f40 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:02.532214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 171/335] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:02.542834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c2d36-a854-4dc6-a0a7-7a1c87ffe3fd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:02.543661 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c2d36-a854-4dc6-a0a7-7a1c87ffe3fd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:02.544095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51c2d36-a854-4dc6-a0a7-7a1c87ffe3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:02.566529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a51c2d36-a854-4dc6-a0a7-7a1c87ffe3fd None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:02.568040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a51c2d36-a854-4dc6-a0a7-7a1c87ffe3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:02.669061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 165/336] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:02.704722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259a475-b7c9-4db5-858e-f006aca99f40 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:02.856003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a259a475-b7c9-4db5-858e-f006aca99f40 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:02.889268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:02.897960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e2f7b-4c3d-4218-8925-1444036755f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:02.898795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e2f7b-4c3d-4218-8925-1444036755f3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:02.899061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e2f7b-4c3d-4218-8925-1444036755f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:02.899344 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e2f7b-4c3d-4218-8925-1444036755f3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:02.918263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e2f7b-4c3d-4218-8925-1444036755f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d34e2f7b-4c3d-4218-8925-1444036755f3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:02.927223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:02.928094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:02.929324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 172/337] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:02.934960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c2d36-a854-4dc6-a0a7-7a1c87ffe3fd None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:03.059251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:03.079876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51c2d36-a854-4dc6-a0a7-7a1c87ffe3fd None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:03.093683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67929148-c82c-40b3-9128-92e6ea5549cd None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:03.094915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67929148-c82c-40b3-9128-92e6ea5549cd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:03.095419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67929148-c82c-40b3-9128-92e6ea5549cd None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:03.097309 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 173/338] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:03.111676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18083324-d047-48b5-9165-c7dd22bf087d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:03.112263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18083324-d047-48b5-9165-c7dd22bf087d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:03.112738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18083324-d047-48b5-9165-c7dd22bf087d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:03.143234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-18083324-d047-48b5-9165-c7dd22bf087d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:03.144761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-18083324-d047-48b5-9165-c7dd22bf087d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:03.150723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b1a40-9c2e-4379-a017-7c0228fb0064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:03.151545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b1a40-9c2e-4379-a017-7c0228fb0064 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:03.151839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b1a40-9c2e-4379-a017-7c0228fb0064 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:03.152120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483b1a40-9c2e-4379-a017-7c0228fb0064 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:03.153001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b1a40-9c2e-4379-a017-7c0228fb0064 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:03.153316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b1a40-9c2e-4379-a017-7c0228fb0064 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:03.154738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483b1a40-9c2e-4379-a017-7c0228fb0064 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:03.169789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 166/339] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:03.177104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b1a40-9c2e-4379-a017-7c0228fb0064 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:03.277312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 174/340] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:03.318187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18083324-d047-48b5-9165-c7dd22bf087d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:03.357006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483b1a40-9c2e-4379-a017-7c0228fb0064 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:03.427675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a335ec18-065d-40f7-91c4-4e3cfb41bdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:03.428255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335ec18-065d-40f7-91c4-4e3cfb41bdfd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:03.428540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335ec18-065d-40f7-91c4-4e3cfb41bdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:03.428816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a335ec18-065d-40f7-91c4-4e3cfb41bdfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:03.465779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:03.509771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18083324-d047-48b5-9165-c7dd22bf087d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:03.549776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:03.561031 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335ec18-065d-40f7-91c4-4e3cfb41bdfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:03.562648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a335ec18-065d-40f7-91c4-4e3cfb41bdfd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:03.572717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f86f68f-4c93-42a6-9833-f930a898000b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:03.573749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86f68f-4c93-42a6-9833-f930a898000b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:03.574236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86f68f-4c93-42a6-9833-f930a898000b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:03.574700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f86f68f-4c93-42a6-9833-f930a898000b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:03.589632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f86f68f-4c93-42a6-9833-f930a898000b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2f86f68f-4c93-42a6-9833-f930a898000b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:03.604286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:03.605324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:03.606275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:03.612189 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 175/341] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:03.614415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7690f-7bce-4e68-bea2-6a567b109855 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:03.616842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7690f-7bce-4e68-bea2-6a567b109855 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:03.617999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7690f-7bce-4e68-bea2-6a567b109855 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:03.621609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 176/342] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:03.623136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7690f-7bce-4e68-bea2-6a567b109855 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:03.742856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 167/343] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:14:03.744862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75521ae-45ad-48fb-8ac0-7088125fec7e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:03.745152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75521ae-45ad-48fb-8ac0-7088125fec7e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:03.745452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75521ae-45ad-48fb-8ac0-7088125fec7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:03.781526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f75521ae-45ad-48fb-8ac0-7088125fec7e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:03.782739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f75521ae-45ad-48fb-8ac0-7088125fec7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:03.817777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7690f-7bce-4e68-bea2-6a567b109855 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:03.856597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:03.866296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d681869-1edd-41d3-9205-f67ef8cb39bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:03.866972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d681869-1edd-41d3-9205-f67ef8cb39bd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:03.867304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d681869-1edd-41d3-9205-f67ef8cb39bd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:03.867582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d681869-1edd-41d3-9205-f67ef8cb39bd None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:03.868342 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d681869-1edd-41d3-9205-f67ef8cb39bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:03.868650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d681869-1edd-41d3-9205-f67ef8cb39bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:03.870826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d681869-1edd-41d3-9205-f67ef8cb39bd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:03.882580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 177/344] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:14:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:03.883463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d681869-1edd-41d3-9205-f67ef8cb39bd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:03.897250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 168/345] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:03.898833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ab39-8ebd-4156-8923-b7702168c4c8 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:03.899123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ab39-8ebd-4156-8923-b7702168c4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:03.899398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6196ab39-8ebd-4156-8923-b7702168c4c8 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:03.900996 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 169/346] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:03.903443 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196ab39-8ebd-4156-8923-b7702168c4c8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:03.955652 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:04.029849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d681869-1edd-41d3-9205-f67ef8cb39bd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:04.045636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6196ab39-8ebd-4156-8923-b7702168c4c8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:04.069307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e624763-2dbb-4796-a313-938aa7df6c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:04.070260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e624763-2dbb-4796-a313-938aa7df6c08 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.070730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e624763-2dbb-4796-a313-938aa7df6c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.071145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e624763-2dbb-4796-a313-938aa7df6c08 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.072474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e624763-2dbb-4796-a313-938aa7df6c08 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:04.073990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e624763-2dbb-4796-a313-938aa7df6c08 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:04.075886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 178/347] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:03 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:04.077024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d59fc-0aee-44fe-a947-c9c647ef301b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.078104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d59fc-0aee-44fe-a947-c9c647ef301b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.078542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d59fc-0aee-44fe-a947-c9c647ef301b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.088169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cadf9-e1ec-44af-9e7a-a22be2f7ef6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:04.089121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cadf9-e1ec-44af-9e7a-a22be2f7ef6e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.089656 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cadf9-e1ec-44af-9e7a-a22be2f7ef6e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.090667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706cadf9-e1ec-44af-9e7a-a22be2f7ef6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.096852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-703d59fc-0aee-44fe-a947-c9c647ef301b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:04.098286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-703d59fc-0aee-44fe-a947-c9c647ef301b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:04.155033 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:04.185313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 179/348] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:04.186675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d59fc-0aee-44fe-a947-c9c647ef301b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:04.192535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706cadf9-e1ec-44af-9e7a-a22be2f7ef6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:04.194167 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706cadf9-e1ec-44af-9e7a-a22be2f7ef6e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:04.244063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:04.327724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d59fc-0aee-44fe-a947-c9c647ef301b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:04.353509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 170/349] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:04.354208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cadf9-e1ec-44af-9e7a-a22be2f7ef6e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:04.374271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9c3a-e41e-49f2-9f4d-a52095e8d0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:04.375870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9c3a-e41e-49f2-9f4d-a52095e8d0d5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.376831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9c3a-e41e-49f2-9f4d-a52095e8d0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.377261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45d9c3a-e41e-49f2-9f4d-a52095e8d0d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.393153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45d9c3a-e41e-49f2-9f4d-a52095e8d0d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e45d9c3a-e41e-49f2-9f4d-a52095e8d0d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:04.403185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:04.404107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 180/350] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:04.405352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9c3a-e41e-49f2-9f4d-a52095e8d0d5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:04.458704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:04.486819 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706cadf9-e1ec-44af-9e7a-a22be2f7ef6e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:04.519697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:04.525284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45d9c3a-e41e-49f2-9f4d-a52095e8d0d5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:04.527235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb1d5f-790b-4a8b-97a9-d262506eb52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:04.528947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb1d5f-790b-4a8b-97a9-d262506eb52b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.529239 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb1d5f-790b-4a8b-97a9-d262506eb52b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.529561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcb1d5f-790b-4a8b-97a9-d262506eb52b None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.540362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb1d5f-790b-4a8b-97a9-d262506eb52b None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:04.541745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcb1d5f-790b-4a8b-97a9-d262506eb52b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:04.543045 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 171/351] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:04 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:04.544138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb1d5f-790b-4a8b-97a9-d262506eb52b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:04.574995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5cf4-f3d2-4e9e-9d52-4fdaedc131a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:04.575824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5cf4-f3d2-4e9e-9d52-4fdaedc131a4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.576266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5cf4-f3d2-4e9e-9d52-4fdaedc131a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.576545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63a5cf4-f3d2-4e9e-9d52-4fdaedc131a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.588957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63a5cf4-f3d2-4e9e-9d52-4fdaedc131a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c63a5cf4-f3d2-4e9e-9d52-4fdaedc131a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:04.598900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:04.599981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:04.600842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:04.605025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 181/352] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:04.606057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5cf4-f3d2-4e9e-9d52-4fdaedc131a4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:04.686617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcb1d5f-790b-4a8b-97a9-d262506eb52b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:04.733511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f6938-ea3e-4eca-849b-6e01b2b32e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:04.735151 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6938-ea3e-4eca-849b-6e01b2b32e13 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.735642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6938-ea3e-4eca-849b-6e01b2b32e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.736249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f6938-ea3e-4eca-849b-6e01b2b32e13 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.737704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f6938-ea3e-4eca-849b-6e01b2b32e13 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:04.739455 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f6938-ea3e-4eca-849b-6e01b2b32e13 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:04.741329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 172/353] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:04 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:04.742526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d69022-435d-40bd-89e0-477df51fa38a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.742832 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63a5cf4-f3d2-4e9e-9d52-4fdaedc131a4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:04.743148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d69022-435d-40bd-89e0-477df51fa38a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.743574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d69022-435d-40bd-89e0-477df51fa38a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.744996 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 173/354] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:04.749688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d69022-435d-40bd-89e0-477df51fa38a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:04.768209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:04.784381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a434d-3835-405a-8a12-b06d2a4d5573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:04.785344 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a434d-3835-405a-8a12-b06d2a4d5573 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.785869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a434d-3835-405a-8a12-b06d2a4d5573 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.786350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307a434d-3835-405a-8a12-b06d2a4d5573 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.787295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a434d-3835-405a-8a12-b06d2a4d5573 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:04.787787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a434d-3835-405a-8a12-b06d2a4d5573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:04.789406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307a434d-3835-405a-8a12-b06d2a4d5573 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:04.799717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 182/355] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:04.800801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a434d-3835-405a-8a12-b06d2a4d5573 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:04.826049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:04.870988 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d69022-435d-40bd-89e0-477df51fa38a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:04.914064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c6e2d-3fb1-4e6b-b14f-a2bd6d08a4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:04.914721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c6e2d-3fb1-4e6b-b14f-a2bd6d08a4f6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.915007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c6e2d-3fb1-4e6b-b14f-a2bd6d08a4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.915281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c6e2d-3fb1-4e6b-b14f-a2bd6d08a4f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.916120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c6e2d-3fb1-4e6b-b14f-a2bd6d08a4f6 None admin] RBAC: Adding query filter params (group.id=ad10e76c7d0042078bb574adb4da63df, role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=23a2e92a012a4c6995bc96295899d286) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:04.916434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c6e2d-3fb1-4e6b-b14f-a2bd6d08a4f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:04.917914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c6e2d-3fb1-4e6b-b14f-a2bd6d08a4f6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:04.922999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307a434d-3835-405a-8a12-b06d2a4d5573 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:04.926206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 174/356] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:04 2022] GET /identity/v3/role_assignments?group.id=ad10e76c7d0042078bb574adb4da63df&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=3b6f494c47a6407284d1d344940e34cd => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:04.927022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c6e2d-3fb1-4e6b-b14f-a2bd6d08a4f6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:04.970202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69178326-f622-4b0f-83f4-33da38b20411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:04.970894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69178326-f622-4b0f-83f4-33da38b20411 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.971179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69178326-f622-4b0f-83f4-33da38b20411 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.971453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69178326-f622-4b0f-83f4-33da38b20411 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:04.972219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69178326-f622-4b0f-83f4-33da38b20411 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:04.972528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69178326-f622-4b0f-83f4-33da38b20411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:04.974066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69178326-f622-4b0f-83f4-33da38b20411 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:04.983571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 183/357] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:04.986093 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ff4a4-616e-4888-86c5-82f9f025ae80 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:04.986576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ff4a4-616e-4888-86c5-82f9f025ae80 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:04.987015 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ff4a4-616e-4888-86c5-82f9f025ae80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:05.007670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d68ff4a4-616e-4888-86c5-82f9f025ae80 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:05.008899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d68ff4a4-616e-4888-86c5-82f9f025ae80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:05.021075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:05.060843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c6e2d-3fb1-4e6b-b14f-a2bd6d08a4f6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:05.105843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d1f5d-bdbc-4d6a-9413-91431e00fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:05.106706 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d1f5d-bdbc-4d6a-9413-91431e00fff2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:05.106997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d1f5d-bdbc-4d6a-9413-91431e00fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:05.107260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d1f5d-bdbc-4d6a-9413-91431e00fff2 None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:05.107580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 184/358] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:05.108430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ff4a4-616e-4888-86c5-82f9f025ae80 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:05.108997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d1f5d-bdbc-4d6a-9413-91431e00fff2 None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:05.110578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d1f5d-bdbc-4d6a-9413-91431e00fff2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:05.112673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 175/359] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:04 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:05.118678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08116d50-5c17-49a7-a754-79055e86b437 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:05.119136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08116d50-5c17-49a7-a754-79055e86b437 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:05.119579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08116d50-5c17-49a7-a754-79055e86b437 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:05.121335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 176/360] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:05.122208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08116d50-5c17-49a7-a754-79055e86b437 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:05.134955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:05.230243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ff4a4-616e-4888-86c5-82f9f025ae80 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:05.253696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08116d50-5c17-49a7-a754-79055e86b437 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:05.278777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d3075-c4ef-4cb2-bab9-3941038c2932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:05.280868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d3075-c4ef-4cb2-bab9-3941038c2932 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:05.281751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d3075-c4ef-4cb2-bab9-3941038c2932 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:05.282369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d3075-c4ef-4cb2-bab9-3941038c2932 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:05.300541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9e8bb2-5af7-4db4-8659-fc504ff278fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:05.301549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e8bb2-5af7-4db4-8659-fc504ff278fd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:05.301979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e8bb2-5af7-4db4-8659-fc504ff278fd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:05.302402 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9e8bb2-5af7-4db4-8659-fc504ff278fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:05.312957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:05.364148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d3075-c4ef-4cb2-bab9-3941038c2932 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:05.365925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1d3075-c4ef-4cb2-bab9-3941038c2932 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:05.383652 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:05.401902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e8bb2-5af7-4db4-8659-fc504ff278fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:05.404612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9e8bb2-5af7-4db4-8659-fc504ff278fd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:05.500655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 185/361] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:05.502480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d3075-c4ef-4cb2-bab9-3941038c2932 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:05.539273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 177/362] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:05.544934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f70a77-c491-4ec3-82a8-28c1a9634684 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:05.545497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f70a77-c491-4ec3-82a8-28c1a9634684 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:05.545969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f70a77-c491-4ec3-82a8-28c1a9634684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:05.565106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-98f70a77-c491-4ec3-82a8-28c1a9634684 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:05.566697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-98f70a77-c491-4ec3-82a8-28c1a9634684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:05.613728 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:05.644230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d3075-c4ef-4cb2-bab9-3941038c2932 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:05.661498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 178/363] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:05.662652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f70a77-c491-4ec3-82a8-28c1a9634684 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:05.664978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:05.687341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924bdff-5d08-4664-b858-6c04d53ac4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:05.688129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924bdff-5d08-4664-b858-6c04d53ac4fe None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:05.688423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924bdff-5d08-4664-b858-6c04d53ac4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:05.688700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924bdff-5d08-4664-b858-6c04d53ac4fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:05.689576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924bdff-5d08-4664-b858-6c04d53ac4fe None admin] RBAC: Adding query filter params (group.id=ad10e76c7d0042078bb574adb4da63df, role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=23a2e92a012a4c6995bc96295899d286) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:05.689890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924bdff-5d08-4664-b858-6c04d53ac4fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:05.691321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924bdff-5d08-4664-b858-6c04d53ac4fe None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:05.699480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 186/364] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:05 2022] GET /identity/v3/role_assignments?group.id=ad10e76c7d0042078bb574adb4da63df&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:05.699480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924bdff-5d08-4664-b858-6c04d53ac4fe None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:05.799702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f70a77-c491-4ec3-82a8-28c1a9634684 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:05.823463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924bdff-5d08-4664-b858-6c04d53ac4fe None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:05.848701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fbd60-ea79-4792-b2fc-19b3af011675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:05.849556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fbd60-ea79-4792-b2fc-19b3af011675 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:05.849842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fbd60-ea79-4792-b2fc-19b3af011675 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:05.850120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fbd60-ea79-4792-b2fc-19b3af011675 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:05.866920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fbd60-ea79-4792-b2fc-19b3af011675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:05.871891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb235f9-4271-433d-a711-e78e6255ceef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:05.873065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb235f9-4271-433d-a711-e78e6255ceef None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:05.873668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb235f9-4271-433d-a711-e78e6255ceef None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:05.874364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb235f9-4271-433d-a711-e78e6255ceef None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-e75fbd60-ea79-4792-b2fc-19b3af011675 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:05.874824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:05.876733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:05.877572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 179/365] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:05.880092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb235f9-4271-433d-a711-e78e6255ceef None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:05.882676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb235f9-4271-433d-a711-e78e6255ceef None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:05.882790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fbd60-ea79-4792-b2fc-19b3af011675 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:05.885050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 187/366] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:05 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:05.886220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb235f9-4271-433d-a711-e78e6255ceef None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:05.920580 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:05.940059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:06.004493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb235f9-4271-433d-a711-e78e6255ceef None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.013252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fbd60-ea79-4792-b2fc-19b3af011675 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.044469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc3799-68e2-4bac-aab8-2a5b78d81a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.045682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc3799-68e2-4bac-aab8-2a5b78d81a6e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.046171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc3799-68e2-4bac-aab8-2a5b78d81a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.046686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fc3799-68e2-4bac-aab8-2a5b78d81a6e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.050689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09345c8e-86a2-4820-a188-6c02d2a310b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.051753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09345c8e-86a2-4820-a188-6c02d2a310b3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09345c8e-86a2-4820-a188-6c02d2a310b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.052602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09345c8e-86a2-4820-a188-6c02d2a310b3 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.054028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09345c8e-86a2-4820-a188-6c02d2a310b3 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:06.055625 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09345c8e-86a2-4820-a188-6c02d2a310b3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:06.057774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 180/367] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:05 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:06.058685 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09345c8e-86a2-4820-a188-6c02d2a310b3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.063612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fc3799-68e2-4bac-aab8-2a5b78d81a6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-91fc3799-68e2-4bac-aab8-2a5b78d81a6e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:06.069213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:06.070140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:06.070913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 188/368] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:06.071548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc3799-68e2-4bac-aab8-2a5b78d81a6e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.175759 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09345c8e-86a2-4820-a188-6c02d2a310b3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.186007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:06.196821 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fc3799-68e2-4bac-aab8-2a5b78d81a6e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.199487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:06.222906 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a99b6d-172f-451c-b7ca-10a49fe17fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.224045 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a99b6d-172f-451c-b7ca-10a49fe17fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.224335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a99b6d-172f-451c-b7ca-10a49fe17fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.224611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a99b6d-172f-451c-b7ca-10a49fe17fb7 None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.225967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a99b6d-172f-451c-b7ca-10a49fe17fb7 None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:06.227573 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a99b6d-172f-451c-b7ca-10a49fe17fb7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:06.229454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 181/369] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:06.230446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a99b6d-172f-451c-b7ca-10a49fe17fb7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.242308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560dfde1-33c9-4ca4-8b46-acf5c57654bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.243055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dfde1-33c9-4ca4-8b46-acf5c57654bb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.243354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dfde1-33c9-4ca4-8b46-acf5c57654bb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.243681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560dfde1-33c9-4ca4-8b46-acf5c57654bb None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.244842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560dfde1-33c9-4ca4-8b46-acf5c57654bb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:06.245167 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560dfde1-33c9-4ca4-8b46-acf5c57654bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:06.246978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560dfde1-33c9-4ca4-8b46-acf5c57654bb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:06.258989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 189/370] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:06.260256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560dfde1-33c9-4ca4-8b46-acf5c57654bb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.381666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a99b6d-172f-451c-b7ca-10a49fe17fb7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.389235 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560dfde1-33c9-4ca4-8b46-acf5c57654bb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.421197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dc2bc-95e3-4ed7-b0ce-43e222fc9e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.423674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dc2bc-95e3-4ed7-b0ce-43e222fc9e55 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.424220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dc2bc-95e3-4ed7-b0ce-43e222fc9e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.424734 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417dc2bc-95e3-4ed7-b0ce-43e222fc9e55 None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.426481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dc2bc-95e3-4ed7-b0ce-43e222fc9e55 None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:06.426893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51d8bd-d95b-4f34-a148-1e3d7a27e7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.427633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d8bd-d95b-4f34-a148-1e3d7a27e7fa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.427986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d8bd-d95b-4f34-a148-1e3d7a27e7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.428334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51d8bd-d95b-4f34-a148-1e3d7a27e7fa None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.428775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417dc2bc-95e3-4ed7-b0ce-43e222fc9e55 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:06.429230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51d8bd-d95b-4f34-a148-1e3d7a27e7fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:06.429628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51d8bd-d95b-4f34-a148-1e3d7a27e7fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:06.430909 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 182/371] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:06.431202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51d8bd-d95b-4f34-a148-1e3d7a27e7fa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:06.431971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dc2bc-95e3-4ed7-b0ce-43e222fc9e55 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.442014 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 190/372] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:06.443211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51d8bd-d95b-4f34-a148-1e3d7a27e7fa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.473902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:06.478047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:06.553226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417dc2bc-95e3-4ed7-b0ce-43e222fc9e55 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.557652 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51d8bd-d95b-4f34-a148-1e3d7a27e7fa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.591641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fff7ec-e95d-4019-b37b-96943a939f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.592664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fff7ec-e95d-4019-b37b-96943a939f4c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.592791 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562e50d-714b-4e3a-b0a6-a5da7dfb00f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.593209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fff7ec-e95d-4019-b37b-96943a939f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.593330 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562e50d-714b-4e3a-b0a6-a5da7dfb00f9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.593740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562e50d-714b-4e3a-b0a6-a5da7dfb00f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.594378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fff7ec-e95d-4019-b37b-96943a939f4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.594789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562e50d-714b-4e3a-b0a6-a5da7dfb00f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.595234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fff7ec-e95d-4019-b37b-96943a939f4c None admin] RBAC: Adding query filter params (group.id=ad10e76c7d0042078bb574adb4da63df, role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:06.595550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fff7ec-e95d-4019-b37b-96943a939f4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:06.597114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fff7ec-e95d-4019-b37b-96943a939f4c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:06.603798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 183/373] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/role_assignments?group.id=ad10e76c7d0042078bb574adb4da63df&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:06.604897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fff7ec-e95d-4019-b37b-96943a939f4c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.689525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562e50d-714b-4e3a-b0a6-a5da7dfb00f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:06.690916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562e50d-714b-4e3a-b0a6-a5da7dfb00f9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:06.710894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:06.717665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fff7ec-e95d-4019-b37b-96943a939f4c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.725591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:06.760011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c18c-eefc-4ad9-86d0-93f772cfd451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.761235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c18c-eefc-4ad9-86d0-93f772cfd451 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.761702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c18c-eefc-4ad9-86d0-93f772cfd451 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.762149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7446c18c-eefc-4ad9-86d0-93f772cfd451 None admin] PATH_INFO: `/v3/users/218b36f90a094ce29641a1acd8e07998` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.763591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446c18c-eefc-4ad9-86d0-93f772cfd451 None admin] RBAC: Authorizing `identity:get_user(user_id=218b36f90a094ce29641a1acd8e07998)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:06.765276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7446c18c-eefc-4ad9-86d0-93f772cfd451 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:06.768310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 184/374] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/users/218b36f90a094ce29641a1acd8e07998 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:06.769258 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c18c-eefc-4ad9-86d0-93f772cfd451 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.839095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 191/375] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:06.840193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562e50d-714b-4e3a-b0a6-a5da7dfb00f9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.884070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7446c18c-eefc-4ad9-86d0-93f772cfd451 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.923570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189d744-386e-45c8-8ff7-8931e25800c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:06.924471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189d744-386e-45c8-8ff7-8931e25800c4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:06.924922 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189d744-386e-45c8-8ff7-8931e25800c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:06.925341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c189d744-386e-45c8-8ff7-8931e25800c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:06.926346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189d744-386e-45c8-8ff7-8931e25800c4 None admin] RBAC: Adding query filter params (group.id=ad10e76c7d0042078bb574adb4da63df, role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:06.926800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189d744-386e-45c8-8ff7-8931e25800c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:06.928504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c189d744-386e-45c8-8ff7-8931e25800c4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:06.935118 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 185/376] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/role_assignments?group.id=ad10e76c7d0042078bb574adb4da63df&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=3b6f494c47a6407284d1d344940e34cd => generated 260 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:06.936346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189d744-386e-45c8-8ff7-8931e25800c4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:06.956332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562e50d-714b-4e3a-b0a6-a5da7dfb00f9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:06.982250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:06.988342 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:07.008206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb4280-e0b3-4dd5-b4ce-bb966c25e854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:07.008904 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb4280-e0b3-4dd5-b4ce-bb966c25e854 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:07.009190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb4280-e0b3-4dd5-b4ce-bb966c25e854 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:07.009491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb4280-e0b3-4dd5-b4ce-bb966c25e854 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:07.010452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb4280-e0b3-4dd5-b4ce-bb966c25e854 None admin] RBAC: Adding query filter params (role.id=4f7a6e180d6d454bb3458ca27d7d822f, user.id=218b36f90a094ce29641a1acd8e07998, scope.system=all) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:07.010772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb4280-e0b3-4dd5-b4ce-bb966c25e854 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:07.012318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bb4280-e0b3-4dd5-b4ce-bb966c25e854 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:07.040985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 192/377] 10.222.0.23 () {56 vars in 1308 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/role_assignments?user.id=218b36f90a094ce29641a1acd8e07998&scope.system=all&role.id=4f7a6e180d6d454bb3458ca27d7d822f => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:07.099345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c189d744-386e-45c8-8ff7-8931e25800c4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:07.155150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeadf0f-a3d1-4e3f-8851-aca236cec31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:07.156040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeadf0f-a3d1-4e3f-8851-aca236cec31f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:07.157019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeadf0f-a3d1-4e3f-8851-aca236cec31f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:07.157505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeadf0f-a3d1-4e3f-8851-aca236cec31f None admin] PATH_INFO: `/v3/projects/65457e2fc15c4d94825d2ac5ee397f82` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:07.158952 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeadf0f-a3d1-4e3f-8851-aca236cec31f None admin] RBAC: Authorizing `identity:get_project(project_id=65457e2fc15c4d94825d2ac5ee397f82)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:07.160468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeadf0f-a3d1-4e3f-8851-aca236cec31f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:07.162731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 186/378] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:06 2022] GET /identity/v3/projects/65457e2fc15c4d94825d2ac5ee397f82 => generated 344 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:07.168048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb4280-e0b3-4dd5-b4ce-bb966c25e854 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:07.285941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bb4280-e0b3-4dd5-b4ce-bb966c25e854 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:07.323230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada8692-f91d-4422-a917-998268dd952f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:07.324099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada8692-f91d-4422-a917-998268dd952f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:07.324381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada8692-f91d-4422-a917-998268dd952f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:07.324654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada8692-f91d-4422-a917-998268dd952f None admin] PATH_INFO: `/v3/groups/67172b0ddcca4d3bbae81117879cfd38` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:07.335631 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada8692-f91d-4422-a917-998268dd952f None admin] RBAC: Authorizing `identity:get_group(group_id=67172b0ddcca4d3bbae81117879cfd38)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:07.337094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada8692-f91d-4422-a917-998268dd952f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:07.339230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 193/379] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:07 2022] GET /identity/v3/groups/67172b0ddcca4d3bbae81117879cfd38 => generated 229 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:07.345536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeadf0f-a3d1-4e3f-8851-aca236cec31f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:07.451782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:07.459195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeadf0f-a3d1-4e3f-8851-aca236cec31f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:07.498201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8153c4-f93b-43bd-b65f-b21d8ad67096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:07.499158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8153c4-f93b-43bd-b65f-b21d8ad67096 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:07.499615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8153c4-f93b-43bd-b65f-b21d8ad67096 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:07.500856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8153c4-f93b-43bd-b65f-b21d8ad67096 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:07.501903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8153c4-f93b-43bd-b65f-b21d8ad67096 None admin] RBAC: Adding query filter params (group.id=67172b0ddcca4d3bbae81117879cfd38, role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=65457e2fc15c4d94825d2ac5ee397f82) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:07.502391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8153c4-f93b-43bd-b65f-b21d8ad67096 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:07.504083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8153c4-f93b-43bd-b65f-b21d8ad67096 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:07.511225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 187/380] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:07 2022] GET /identity/v3/role_assignments?group.id=67172b0ddcca4d3bbae81117879cfd38&scope.project.id=65457e2fc15c4d94825d2ac5ee397f82&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:07.552945 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832d8c7-3b61-4f2c-8e0d-3c26cbe01a4b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:07.553553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832d8c7-3b61-4f2c-8e0d-3c26cbe01a4b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:07.554007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832d8c7-3b61-4f2c-8e0d-3c26cbe01a4b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:07.555628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 194/381] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:07.561842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086841e-5f07-44ad-a3e9-665a0d6a8b54 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:07.563088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086841e-5f07-44ad-a3e9-665a0d6a8b54 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:07.563603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c086841e-5f07-44ad-a3e9-665a0d6a8b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:07.587389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c086841e-5f07-44ad-a3e9-665a0d6a8b54 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:07.588932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c086841e-5f07-44ad-a3e9-665a0d6a8b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:07.673212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 188/382] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:07.698112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832d8c7-3b61-4f2c-8e0d-3c26cbe01a4b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:07.762989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:07.827871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832d8c7-3b61-4f2c-8e0d-3c26cbe01a4b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:07.866434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436ed89-ba3a-4f8f-b7bd-ab2572bcec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:07.867295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436ed89-ba3a-4f8f-b7bd-ab2572bcec3b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:07.867580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436ed89-ba3a-4f8f-b7bd-ab2572bcec3b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:07.867854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436ed89-ba3a-4f8f-b7bd-ab2572bcec3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:07.879454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436ed89-ba3a-4f8f-b7bd-ab2572bcec3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7436ed89-ba3a-4f8f-b7bd-ab2572bcec3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:07.887935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:07.890534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:07.891380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 195/383] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:07.897017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086841e-5f07-44ad-a3e9-665a0d6a8b54 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:07.996294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:08.006441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c086841e-5f07-44ad-a3e9-665a0d6a8b54 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:08.044318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ae4d6-ec37-4a35-b63e-02b14d240163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:08.044914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ae4d6-ec37-4a35-b63e-02b14d240163 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:08.045199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ae4d6-ec37-4a35-b63e-02b14d240163 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:08.045499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1ae4d6-ec37-4a35-b63e-02b14d240163 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:08.046248 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ae4d6-ec37-4a35-b63e-02b14d240163 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:08.046551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ae4d6-ec37-4a35-b63e-02b14d240163 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:08.048161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1ae4d6-ec37-4a35-b63e-02b14d240163 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:08.056928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 189/384] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:08.062777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436ed89-ba3a-4f8f-b7bd-ab2572bcec3b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:08.173146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436ed89-ba3a-4f8f-b7bd-ab2572bcec3b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:08.186414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:08.225427 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fcb14f-d317-43b9-abdc-04fcf99c266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:08.226326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcb14f-d317-43b9-abdc-04fcf99c266b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:08.226611 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcb14f-d317-43b9-abdc-04fcf99c266b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:08.226887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fcb14f-d317-43b9-abdc-04fcf99c266b None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:08.228113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcb14f-d317-43b9-abdc-04fcf99c266b None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:08.229577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fcb14f-d317-43b9-abdc-04fcf99c266b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:08.231510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 196/385] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:08 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:08.237017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ae4d6-ec37-4a35-b63e-02b14d240163 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:08.318582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817629d5-62c7-46c6-9583-f886a53df83a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:08.318764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817629d5-62c7-46c6-9583-f886a53df83a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:08.319068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817629d5-62c7-46c6-9583-f886a53df83a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:08.320670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 197/386] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:08.325624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36284980-d72d-4fe6-90eb-1f5cc8edb68e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:08.325911 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36284980-d72d-4fe6-90eb-1f5cc8edb68e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:08.326192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36284980-d72d-4fe6-90eb-1f5cc8edb68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:08.346940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1ae4d6-ec37-4a35-b63e-02b14d240163 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:08.347445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-36284980-d72d-4fe6-90eb-1f5cc8edb68e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:08.348659 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-36284980-d72d-4fe6-90eb-1f5cc8edb68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:08.382053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f38d1-90cf-4669-a8a0-4add1ecf9877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:08.382930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f38d1-90cf-4669-a8a0-4add1ecf9877 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:08.383208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f38d1-90cf-4669-a8a0-4add1ecf9877 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:08.383490 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1f38d1-90cf-4669-a8a0-4add1ecf9877 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:08.384683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f38d1-90cf-4669-a8a0-4add1ecf9877 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:08.386166 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1f38d1-90cf-4669-a8a0-4add1ecf9877 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:08.393091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 190/387] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:08 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:08.394169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f38d1-90cf-4669-a8a0-4add1ecf9877 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:08.419266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:08.438387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 198/388] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:08.467934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36284980-d72d-4fe6-90eb-1f5cc8edb68e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:08.553413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1f38d1-90cf-4669-a8a0-4add1ecf9877 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:08.602583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb50aff-76d4-4581-b50c-c16991dc563e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:08.603814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb50aff-76d4-4581-b50c-c16991dc563e None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:08.604309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb50aff-76d4-4581-b50c-c16991dc563e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:08.604870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb50aff-76d4-4581-b50c-c16991dc563e None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286/groups/ad10e76c7d0042078bb574adb4da63df/roles/3b6f494c47a6407284d1d344940e34cd` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:08.607308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb50aff-76d4-4581-b50c-c16991dc563e None admin] RBAC: Authorizing `identity:create_grant(project_id=23a2e92a012a4c6995bc96295899d286, group_id=ad10e76c7d0042078bb574adb4da63df, role_id=3b6f494c47a6407284d1d344940e34cd)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:08.608938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb50aff-76d4-4581-b50c-c16991dc563e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:08.627170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 191/389] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 12 13:14:08 2022] PUT /identity/v3/projects/23a2e92a012a4c6995bc96295899d286/groups/ad10e76c7d0042078bb574adb4da63df/roles/3b6f494c47a6407284d1d344940e34cd => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:14:08.666860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36284980-d72d-4fe6-90eb-1f5cc8edb68e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:08.678612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:08.711320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d39af-063c-4f0e-b754-0040b3775512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:08.712135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d39af-063c-4f0e-b754-0040b3775512 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:08.712421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d39af-063c-4f0e-b754-0040b3775512 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:08.712698 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709d39af-063c-4f0e-b754-0040b3775512 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:08.728213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709d39af-063c-4f0e-b754-0040b3775512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-709d39af-063c-4f0e-b754-0040b3775512 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:08.736279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:08.737131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:08.737934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:08.740217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 199/390] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:08.746977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb50aff-76d4-4581-b50c-c16991dc563e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:08.855259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:08.875708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb50aff-76d4-4581-b50c-c16991dc563e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:08.912580 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56394cfe-b4a8-499f-b42a-61f8798b291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:08.914012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56394cfe-b4a8-499f-b42a-61f8798b291b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:08.914447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56394cfe-b4a8-499f-b42a-61f8798b291b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:08.914865 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56394cfe-b4a8-499f-b42a-61f8798b291b None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:08.915802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56394cfe-b4a8-499f-b42a-61f8798b291b None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:08.916264 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56394cfe-b4a8-499f-b42a-61f8798b291b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:08.917854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56394cfe-b4a8-499f-b42a-61f8798b291b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:08.928018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 192/391] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:14:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:08.934502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d39af-063c-4f0e-b754-0040b3775512 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:09.076628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709d39af-063c-4f0e-b754-0040b3775512 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:09.123477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc0e80-3b52-40d3-ba46-937b37ec8a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:09.124261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0e80-3b52-40d3-ba46-937b37ec8a8f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:09.124548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0e80-3b52-40d3-ba46-937b37ec8a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:09.125353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0e80-3b52-40d3-ba46-937b37ec8a8f None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:09.126587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc0e80-3b52-40d3-ba46-937b37ec8a8f None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:09.128169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc0e80-3b52-40d3-ba46-937b37ec8a8f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:09.129987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 200/392] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:08 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:09.135860 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56394cfe-b4a8-499f-b42a-61f8798b291b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:09.311618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56394cfe-b4a8-499f-b42a-61f8798b291b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:09.350277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c8d1b-b235-44c2-977d-d490e08d8ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:09.351152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c8d1b-b235-44c2-977d-d490e08d8ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:09.351438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c8d1b-b235-44c2-977d-d490e08d8ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:09.351718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c8d1b-b235-44c2-977d-d490e08d8ef8 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:09.352917 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c8d1b-b235-44c2-977d-d490e08d8ef8 None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:09.354392 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c8d1b-b235-44c2-977d-d490e08d8ef8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:09.356458 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 193/393] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:09 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:09.362579 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc0e80-3b52-40d3-ba46-937b37ec8a8f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:09.438902 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:09.536629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc0e80-3b52-40d3-ba46-937b37ec8a8f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:09.582787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38f102-5cf0-442c-bef0-20394b6e7db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:09.584102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38f102-5cf0-442c-bef0-20394b6e7db0 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:09.584558 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38f102-5cf0-442c-bef0-20394b6e7db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:09.584979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38f102-5cf0-442c-bef0-20394b6e7db0 None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286/groups/ad10e76c7d0042078bb574adb4da63df/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:09.589733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38f102-5cf0-442c-bef0-20394b6e7db0 None admin] RBAC: Authorizing `identity:create_grant(project_id=23a2e92a012a4c6995bc96295899d286, group_id=ad10e76c7d0042078bb574adb4da63df, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:09.591346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38f102-5cf0-442c-bef0-20394b6e7db0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:09.625440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 201/394] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 12 13:14:09 2022] PUT /identity/v3/projects/23a2e92a012a4c6995bc96295899d286/groups/ad10e76c7d0042078bb574adb4da63df/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:09.754620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247fe55-2d4e-4d9b-bb47-b612d7baf85e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:09.755153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247fe55-2d4e-4d9b-bb47-b612d7baf85e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:09.755590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8247fe55-2d4e-4d9b-bb47-b612d7baf85e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:09.757684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 194/395] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:09.762942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5adb7a-95a7-43d7-8a3c-eaf83071634f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:09.763912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5adb7a-95a7-43d7-8a3c-eaf83071634f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:09.764448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5adb7a-95a7-43d7-8a3c-eaf83071634f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:09.789464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0a5adb7a-95a7-43d7-8a3c-eaf83071634f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:09.789464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0a5adb7a-95a7-43d7-8a3c-eaf83071634f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:09.867874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 202/396] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:09.892285 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247fe55-2d4e-4d9b-bb47-b612d7baf85e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:09.905834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:10.024195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8247fe55-2d4e-4d9b-bb47-b612d7baf85e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:10.028506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4afe-1d4e-4e8b-bf59-e5884ac18fd0 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.028792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4afe-1d4e-4e8b-bf59-e5884ac18fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.029068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894b4afe-1d4e-4e8b-bf59-e5884ac18fd0 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.030706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 203/397] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:14:10.036249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e0f64-bdbf-41c7-a792-924a843ca392 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.036536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e0f64-bdbf-41c7-a792-924a843ca392 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.036814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459e0f64-bdbf-41c7-a792-924a843ca392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.053763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-459e0f64-bdbf-41c7-a792-924a843ca392 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:10.054952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-459e0f64-bdbf-41c7-a792-924a843ca392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:10.079741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3eaba-fc6f-476d-86e2-6935f7974715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:10.080885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3eaba-fc6f-476d-86e2-6935f7974715 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.081188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3eaba-fc6f-476d-86e2-6935f7974715 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.082047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d3eaba-fc6f-476d-86e2-6935f7974715 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.082173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:10.094149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d3eaba-fc6f-476d-86e2-6935f7974715 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-a7d3eaba-fc6f-476d-86e2-6935f7974715 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:10.104772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:10.105791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:10.106572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:10.110798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 195/398] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:10.118078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3eaba-fc6f-476d-86e2-6935f7974715 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:10.197422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 204/399] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:10.213981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:10.217433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93111c-d458-4910-9ec4-81c2d9e14326 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.218483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93111c-d458-4910-9ec4-81c2d9e14326 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.218483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93111c-d458-4910-9ec4-81c2d9e14326 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.221206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 205/400] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:10.227136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebc008-224f-4dca-8f63-49fb9c83b284 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.227773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebc008-224f-4dca-8f63-49fb9c83b284 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.228250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebc008-224f-4dca-8f63-49fb9c83b284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.249363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-25ebc008-224f-4dca-8f63-49fb9c83b284 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:10.251872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-25ebc008-224f-4dca-8f63-49fb9c83b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:10.277766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d3eaba-fc6f-476d-86e2-6935f7974715 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:10.325857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4844171-7be5-4312-8e75-845d4246d196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:10.326565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4844171-7be5-4312-8e75-845d4246d196 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.326858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4844171-7be5-4312-8e75-845d4246d196 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.327145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4844171-7be5-4312-8e75-845d4246d196 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.333394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4844171-7be5-4312-8e75-845d4246d196 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:10.333394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4844171-7be5-4312-8e75-845d4246d196 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:10.333394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4844171-7be5-4312-8e75-845d4246d196 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:10.333808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 206/401] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:10.334935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebc008-224f-4dca-8f63-49fb9c83b284 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:10.340889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 196/402] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:10.347687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4844171-7be5-4312-8e75-845d4246d196 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:10.443237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:10.461999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebc008-224f-4dca-8f63-49fb9c83b284 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:10.476486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4844171-7be5-4312-8e75-845d4246d196 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:10.506721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97501cf9-6ee9-4b9e-ab8b-44b308309daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:10.507792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97501cf9-6ee9-4b9e-ab8b-44b308309daa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.508084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97501cf9-6ee9-4b9e-ab8b-44b308309daa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.508367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97501cf9-6ee9-4b9e-ab8b-44b308309daa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.510209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:10.516206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ec79-8055-4830-b625-59ebb788bd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:10.517827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ec79-8055-4830-b625-59ebb788bd24 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.518612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ec79-8055-4830-b625-59ebb788bd24 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.519109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ec79-8055-4830-b625-59ebb788bd24 None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.520483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97501cf9-6ee9-4b9e-ab8b-44b308309daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:10.520690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613ec79-8055-4830-b625-59ebb788bd24 None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:10.522351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613ec79-8055-4830-b625-59ebb788bd24 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:10.524182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 197/403] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:10 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-97501cf9-6ee9-4b9e-ab8b-44b308309daa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:10.528935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:10.530005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:10.530789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:10.533590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ec79-8055-4830-b625-59ebb788bd24 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:10.535929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 207/404] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:10.537073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97501cf9-6ee9-4b9e-ab8b-44b308309daa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:10.645753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ec79-8055-4830-b625-59ebb788bd24 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:10.652836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97501cf9-6ee9-4b9e-ab8b-44b308309daa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:10.686031 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dbdf1-5e56-4316-90e8-2ee3ec530287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:10.687228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dbdf1-5e56-4316-90e8-2ee3ec530287 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.688253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dbdf1-5e56-4316-90e8-2ee3ec530287 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.688750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860dbdf1-5e56-4316-90e8-2ee3ec530287 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.689656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03fdc-47db-4791-adf4-c304a77b70db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:10.690609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03fdc-47db-4791-adf4-c304a77b70db None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.690967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03fdc-47db-4791-adf4-c304a77b70db None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.691323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b03fdc-47db-4791-adf4-c304a77b70db None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.692605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03fdc-47db-4791-adf4-c304a77b70db None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:10.694175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b03fdc-47db-4791-adf4-c304a77b70db None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:10.700564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 208/405] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:10 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:10.700564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03fdc-47db-4791-adf4-c304a77b70db None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:10.708163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860dbdf1-5e56-4316-90e8-2ee3ec530287 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-860dbdf1-5e56-4316-90e8-2ee3ec530287 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:10.717805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:10.718710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:10.719650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 198/406] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:10.720909 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dbdf1-5e56-4316-90e8-2ee3ec530287 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:10.721839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:10.802956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:10.835799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b03fdc-47db-4791-adf4-c304a77b70db None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:10.863465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860dbdf1-5e56-4316-90e8-2ee3ec530287 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:10.878252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcb5ba-dbef-472a-a334-9cf0bac49366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:10.879405 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcb5ba-dbef-472a-a334-9cf0bac49366 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.879832 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcb5ba-dbef-472a-a334-9cf0bac49366 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.880260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcb5ba-dbef-472a-a334-9cf0bac49366 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.881670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcb5ba-dbef-472a-a334-9cf0bac49366 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:10.882149 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcb5ba-dbef-472a-a334-9cf0bac49366 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:10.884132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcb5ba-dbef-472a-a334-9cf0bac49366 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:10.896545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 209/407] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:10.897597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcb5ba-dbef-472a-a334-9cf0bac49366 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:10.914208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc7287-9667-4521-b00c-be7a7fe8d8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:10.915106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc7287-9667-4521-b00c-be7a7fe8d8c1 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:10.915396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc7287-9667-4521-b00c-be7a7fe8d8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:10.915734 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cc7287-9667-4521-b00c-be7a7fe8d8c1 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/groups/ad10e76c7d0042078bb574adb4da63df/roles/3b6f494c47a6407284d1d344940e34cd` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:10.918019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc7287-9667-4521-b00c-be7a7fe8d8c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=8faaa2ee2ac54d04b66c0dc66612d84b, group_id=ad10e76c7d0042078bb574adb4da63df, role_id=3b6f494c47a6407284d1d344940e34cd)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:10.919477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cc7287-9667-4521-b00c-be7a7fe8d8c1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:10.940228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 199/408] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 12 13:14:10 2022] PUT /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/groups/ad10e76c7d0042078bb574adb4da63df/roles/3b6f494c47a6407284d1d344940e34cd => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:10.941396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc7287-9667-4521-b00c-be7a7fe8d8c1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:10.992789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:11.018497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcb5ba-dbef-472a-a334-9cf0bac49366 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:11.055388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa7963-2890-43e8-8372-6d530bf06baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:11.056145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa7963-2890-43e8-8372-6d530bf06baa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.056534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa7963-2890-43e8-8372-6d530bf06baa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.056914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55aa7963-2890-43e8-8372-6d530bf06baa None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.057828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa7963-2890-43e8-8372-6d530bf06baa None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:11.058291 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa7963-2890-43e8-8372-6d530bf06baa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:11.059822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55aa7963-2890-43e8-8372-6d530bf06baa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:11.070242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 210/409] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:14:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:11.074311 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cc7287-9667-4521-b00c-be7a7fe8d8c1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:11.079715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa7963-2890-43e8-8372-6d530bf06baa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:11.114529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:11.142484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394a0f6-367c-4973-81ba-33cf7b0904e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:11.143385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394a0f6-367c-4973-81ba-33cf7b0904e9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.143687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394a0f6-367c-4973-81ba-33cf7b0904e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.143975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2394a0f6-367c-4973-81ba-33cf7b0904e9 None admin] PATH_INFO: `/v3/groups/67172b0ddcca4d3bbae81117879cfd38` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.146969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394a0f6-367c-4973-81ba-33cf7b0904e9 None admin] RBAC: Authorizing `identity:get_group(group_id=67172b0ddcca4d3bbae81117879cfd38)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:11.150077 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2394a0f6-367c-4973-81ba-33cf7b0904e9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:11.151823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 200/410] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:10 2022] GET /identity/v3/groups/67172b0ddcca4d3bbae81117879cfd38 => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:11.167095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394a0f6-367c-4973-81ba-33cf7b0904e9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:11.249872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55aa7963-2890-43e8-8372-6d530bf06baa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:11.289012 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1022c4b-69a1-4780-9a59-e38c47d47ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:11.289907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1022c4b-69a1-4780-9a59-e38c47d47ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.290193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1022c4b-69a1-4780-9a59-e38c47d47ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.290555 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1022c4b-69a1-4780-9a59-e38c47d47ca0 None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.291739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1022c4b-69a1-4780-9a59-e38c47d47ca0 None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:11.293472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1022c4b-69a1-4780-9a59-e38c47d47ca0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:11.295168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 211/411] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:11 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:11.300947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8550bb2-245e-4905-92fe-d7c8ac42bcde None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.300947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8550bb2-245e-4905-92fe-d7c8ac42bcde None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.300947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8550bb2-245e-4905-92fe-d7c8ac42bcde None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.300947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 212/412] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:11.301786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8550bb2-245e-4905-92fe-d7c8ac42bcde None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:11.309609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2394a0f6-367c-4973-81ba-33cf7b0904e9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:11.315050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:11.348388 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87549c85-a4fe-4bc0-89e3-175b5da732f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:11.349241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87549c85-a4fe-4bc0-89e3-175b5da732f5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.349559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87549c85-a4fe-4bc0-89e3-175b5da732f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.349829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87549c85-a4fe-4bc0-89e3-175b5da732f5 None admin] PATH_INFO: `/v3/projects/65457e2fc15c4d94825d2ac5ee397f82` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.351029 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87549c85-a4fe-4bc0-89e3-175b5da732f5 None admin] RBAC: Authorizing `identity:get_project(project_id=65457e2fc15c4d94825d2ac5ee397f82)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:11.352487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87549c85-a4fe-4bc0-89e3-175b5da732f5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:11.354733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 201/413] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:11 2022] GET /identity/v3/projects/65457e2fc15c4d94825d2ac5ee397f82 => generated 344 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:11.355885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e9e16-703d-4182-ae85-5a4704cde85e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.356172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e9e16-703d-4182-ae85-5a4704cde85e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.356453 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e9e16-703d-4182-ae85-5a4704cde85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.378484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-955e9e16-703d-4182-ae85-5a4704cde85e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:11.379723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-955e9e16-703d-4182-ae85-5a4704cde85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:11.437384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8550bb2-245e-4905-92fe-d7c8ac42bcde None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:11.440800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:11.469644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 202/414] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:11.469644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e9e16-703d-4182-ae85-5a4704cde85e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:11.488995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd110bb-c1c0-4554-8b64-a230ed1a9ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:11.489912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd110bb-c1c0-4554-8b64-a230ed1a9ccf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.490194 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd110bb-c1c0-4554-8b64-a230ed1a9ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.490469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd110bb-c1c0-4554-8b64-a230ed1a9ccf None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.491815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd110bb-c1c0-4554-8b64-a230ed1a9ccf None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:11.493266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd110bb-c1c0-4554-8b64-a230ed1a9ccf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:11.496574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 213/415] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:11 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:11.497220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd110bb-c1c0-4554-8b64-a230ed1a9ccf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:11.599957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e9e16-703d-4182-ae85-5a4704cde85e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:11.610901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:11.636962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe377d-fbb0-48b0-98e5-7b09fa911e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:11.638090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe377d-fbb0-48b0-98e5-7b09fa911e67 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.638379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe377d-fbb0-48b0-98e5-7b09fa911e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.638658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe377d-fbb0-48b0-98e5-7b09fa911e67 None admin] PATH_INFO: `/v3/projects/65457e2fc15c4d94825d2ac5ee397f82/groups/67172b0ddcca4d3bbae81117879cfd38/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.641222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe377d-fbb0-48b0-98e5-7b09fa911e67 None admin] RBAC: Authorizing `identity:create_grant(project_id=65457e2fc15c4d94825d2ac5ee397f82, group_id=67172b0ddcca4d3bbae81117879cfd38, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:11.642197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd110bb-c1c0-4554-8b64-a230ed1a9ccf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:11.643144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fe377d-fbb0-48b0-98e5-7b09fa911e67 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:11.669672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 203/416] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 12 13:14:11 2022] PUT /identity/v3/projects/65457e2fc15c4d94825d2ac5ee397f82/groups/67172b0ddcca4d3bbae81117879cfd38/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:11.671134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe377d-fbb0-48b0-98e5-7b09fa911e67 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:11.683176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb8ec-1bba-49dc-b557-29b3bfdafb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:11.684355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb8ec-1bba-49dc-b557-29b3bfdafb72 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.684906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb8ec-1bba-49dc-b557-29b3bfdafb72 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.685416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7cb8ec-1bba-49dc-b557-29b3bfdafb72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.698346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7cb8ec-1bba-49dc-b557-29b3bfdafb72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-be7cb8ec-1bba-49dc-b557-29b3bfdafb72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:11.708147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:11.709121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:11.710307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:11.713048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 214/417] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:11.719666 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb8ec-1bba-49dc-b557-29b3bfdafb72 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:11.755473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:11.820613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe377d-fbb0-48b0-98e5-7b09fa911e67 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:11.858692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccd83c-b23c-42d1-8175-be77474860e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:11.860254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccd83c-b23c-42d1-8175-be77474860e9 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.860454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccd83c-b23c-42d1-8175-be77474860e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.860692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dccd83c-b23c-42d1-8175-be77474860e9 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/groups/ad10e76c7d0042078bb574adb4da63df/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.868987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccd83c-b23c-42d1-8175-be77474860e9 None admin] RBAC: Authorizing `identity:create_grant(project_id=8faaa2ee2ac54d04b66c0dc66612d84b, group_id=ad10e76c7d0042078bb574adb4da63df, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:11.870840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dccd83c-b23c-42d1-8175-be77474860e9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:11.887395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7cb8ec-1bba-49dc-b557-29b3bfdafb72 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:11.899593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 204/418] 10.222.0.23 () {58 vars in 1442 bytes} [Tue Jul 12 13:14:11 2022] PUT /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b/groups/ad10e76c7d0042078bb574adb4da63df/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:11.943930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d7d3d5-b231-4dd9-832e-569c01ee568c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:11.945070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7d3d5-b231-4dd9-832e-569c01ee568c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.945663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7d3d5-b231-4dd9-832e-569c01ee568c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.946170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d7d3d5-b231-4dd9-832e-569c01ee568c None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:11.947188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7d3d5-b231-4dd9-832e-569c01ee568c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:11.947698 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7d3d5-b231-4dd9-832e-569c01ee568c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:11.949388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d7d3d5-b231-4dd9-832e-569c01ee568c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:11.951395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:11.960071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 215/419] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:11.965483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccd83c-b23c-42d1-8175-be77474860e9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:11.998677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df31222-e99b-4eab-8316-7d4fafa87768 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:11.999044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df31222-e99b-4eab-8316-7d4fafa87768 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:11.999394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df31222-e99b-4eab-8316-7d4fafa87768 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:12.001868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 216/420] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:14:12.009443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac8b2-057a-46b6-91b6-856f417d3a4c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:12.010002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac8b2-057a-46b6-91b6-856f417d3a4c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:12.010452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ac8b2-057a-46b6-91b6-856f417d3a4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:12.033068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d77ac8b2-057a-46b6-91b6-856f417d3a4c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:12.034729 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d77ac8b2-057a-46b6-91b6-856f417d3a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:12.104487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dccd83c-b23c-42d1-8175-be77474860e9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:12.115515 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 217/421] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:12.126276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:12.140597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac8b2-057a-46b6-91b6-856f417d3a4c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:12.145574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e9d6e-1d64-4b00-962f-5da0119a3340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:12.146244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e9d6e-1d64-4b00-962f-5da0119a3340 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:12.146534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e9d6e-1d64-4b00-962f-5da0119a3340 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:12.146814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72e9d6e-1d64-4b00-962f-5da0119a3340 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:12.243168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e9d6e-1d64-4b00-962f-5da0119a3340 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:12.244673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72e9d6e-1d64-4b00-962f-5da0119a3340 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:12.265339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ac8b2-057a-46b6-91b6-856f417d3a4c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:12.305044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:12.318058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a23960-de98-48ec-a49c-08bc26c1f159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:12.319032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a23960-de98-48ec-a49c-08bc26c1f159 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:12.319413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a23960-de98-48ec-a49c-08bc26c1f159 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:12.319700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a23960-de98-48ec-a49c-08bc26c1f159 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:12.333411 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a23960-de98-48ec-a49c-08bc26c1f159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f9a23960-de98-48ec-a49c-08bc26c1f159 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:12.344794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:12.345737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:12.346526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 218/422] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:12.355921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a23960-de98-48ec-a49c-08bc26c1f159 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:12.380314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 205/423] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:12.385657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e9d6e-1d64-4b00-962f-5da0119a3340 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:12.401486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:12.531566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a23960-de98-48ec-a49c-08bc26c1f159 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:12.568660 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9acf4a-06d2-4cc1-91a5-919dbd439b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:12.569719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9acf4a-06d2-4cc1-91a5-919dbd439b52 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:12.570200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9acf4a-06d2-4cc1-91a5-919dbd439b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:12.570961 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9acf4a-06d2-4cc1-91a5-919dbd439b52 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:12.571910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9acf4a-06d2-4cc1-91a5-919dbd439b52 None admin] RBAC: Adding query filter params (name=member) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:12.572429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9acf4a-06d2-4cc1-91a5-919dbd439b52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:12.574630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9acf4a-06d2-4cc1-91a5-919dbd439b52 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:12.584409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 219/424] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:14:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:12.590406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9acf4a-06d2-4cc1-91a5-919dbd439b52 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:12.622568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72e9d6e-1d64-4b00-962f-5da0119a3340 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:12.653814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:12.673366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aee022-8102-4979-903e-f44ae22e3b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:12.674264 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aee022-8102-4979-903e-f44ae22e3b8e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:12.674637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aee022-8102-4979-903e-f44ae22e3b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:12.674923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aee022-8102-4979-903e-f44ae22e3b8e None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:12.676147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aee022-8102-4979-903e-f44ae22e3b8e None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:12.677641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aee022-8102-4979-903e-f44ae22e3b8e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:12.679701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 206/425] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:12 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:12.684828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aee022-8102-4979-903e-f44ae22e3b8e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:12.712850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9acf4a-06d2-4cc1-91a5-919dbd439b52 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:12.777123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:12.790524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e13b2-5d50-4aec-9c59-7f010eff1154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:12.791186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e13b2-5d50-4aec-9c59-7f010eff1154 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:12.791472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e13b2-5d50-4aec-9c59-7f010eff1154 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:12.791748 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1e13b2-5d50-4aec-9c59-7f010eff1154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:12.806863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aee022-8102-4979-903e-f44ae22e3b8e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:12.848612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df350e-28cc-4bbf-bb98-c4c175cad595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:12.849672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df350e-28cc-4bbf-bb98-c4c175cad595 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:12.850299 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df350e-28cc-4bbf-bb98-c4c175cad595 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:12.850683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df350e-28cc-4bbf-bb98-c4c175cad595 None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:12.852004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df350e-28cc-4bbf-bb98-c4c175cad595 None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:12.853579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04df350e-28cc-4bbf-bb98-c4c175cad595 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:12.856309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 207/426] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:12 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:12.862611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df350e-28cc-4bbf-bb98-c4c175cad595 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:12.989344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df350e-28cc-4bbf-bb98-c4c175cad595 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:12.998467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e13b2-5d50-4aec-9c59-7f010eff1154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:13.000554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1e13b2-5d50-4aec-9c59-7f010eff1154 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:13.026837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e424df-65d8-4b6f-b443-9f7fa719c9b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:13.027505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e424df-65d8-4b6f-b443-9f7fa719c9b4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:13.027793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e424df-65d8-4b6f-b443-9f7fa719c9b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:13.028078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e424df-65d8-4b6f-b443-9f7fa719c9b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:13.028929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e424df-65d8-4b6f-b443-9f7fa719c9b4 None admin] RBAC: Adding query filter params (group.id=ad10e76c7d0042078bb574adb4da63df, role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=23a2e92a012a4c6995bc96295899d286) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:13.029244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e424df-65d8-4b6f-b443-9f7fa719c9b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:13.030719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e424df-65d8-4b6f-b443-9f7fa719c9b4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:13.038577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 208/427] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:12 2022] GET /identity/v3/role_assignments?group.id=ad10e76c7d0042078bb574adb4da63df&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=3b6f494c47a6407284d1d344940e34cd => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:13.069835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:13.124864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 220/428] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:13.130459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e424df-65d8-4b6f-b443-9f7fa719c9b4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:13.150214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:13.293948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e424df-65d8-4b6f-b443-9f7fa719c9b4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:13.330456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbcebc-c150-4353-b2b0-25830152f86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:13.331327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbcebc-c150-4353-b2b0-25830152f86b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:13.331601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbcebc-c150-4353-b2b0-25830152f86b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:13.331877 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbcebc-c150-4353-b2b0-25830152f86b None admin] PATH_INFO: `/v3/projects/23a2e92a012a4c6995bc96295899d286` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:13.333055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbcebc-c150-4353-b2b0-25830152f86b None admin] RBAC: Authorizing `identity:get_project(project_id=23a2e92a012a4c6995bc96295899d286)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:13.334521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbcebc-c150-4353-b2b0-25830152f86b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:13.336514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 209/429] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:13 2022] GET /identity/v3/projects/23a2e92a012a4c6995bc96295899d286 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:13.342170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e13b2-5d50-4aec-9c59-7f010eff1154 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:13.419138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c8480-41b3-48a4-b7fc-18675ded6b3f None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:13.419474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c8480-41b3-48a4-b7fc-18675ded6b3f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:13.419694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210c8480-41b3-48a4-b7fc-18675ded6b3f None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:13.421057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 210/430] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:14:13.426903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f3c20-b26f-4a5e-9ae2-aef7bcd41d80 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:13.427175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f3c20-b26f-4a5e-9ae2-aef7bcd41d80 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:13.427454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2f3c20-b26f-4a5e-9ae2-aef7bcd41d80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:13.449674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8d2f3c20-b26f-4a5e-9ae2-aef7bcd41d80 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:13.450946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8d2f3c20-b26f-4a5e-9ae2-aef7bcd41d80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:13.471643 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1e13b2-5d50-4aec-9c59-7f010eff1154 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:13.515518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9c8ad-0738-4be2-b9a5-e75f6a821696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:13.516305 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9c8ad-0738-4be2-b9a5-e75f6a821696 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:13.516586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9c8ad-0738-4be2-b9a5-e75f6a821696 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:13.517151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c9c8ad-0738-4be2-b9a5-e75f6a821696 None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:13.518418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9c8ad-0738-4be2-b9a5-e75f6a821696 None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:13.520710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c9c8ad-0738-4be2-b9a5-e75f6a821696 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:13.523365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 221/431] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:13 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:13.529335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9c8ad-0738-4be2-b9a5-e75f6a821696 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:13.544420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 211/432] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:13.558012 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:13.569535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f3c20-b26f-4a5e-9ae2-aef7bcd41d80 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:13.573637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:13.662784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c9c8ad-0738-4be2-b9a5-e75f6a821696 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:13.688112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2f3c20-b26f-4a5e-9ae2-aef7bcd41d80 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:13.708955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194f963-7342-4091-9b39-e3ff3b98b726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:13.710428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194f963-7342-4091-9b39-e3ff3b98b726 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:13.710915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194f963-7342-4091-9b39-e3ff3b98b726 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:13.711651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194f963-7342-4091-9b39-e3ff3b98b726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:13.712946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194f963-7342-4091-9b39-e3ff3b98b726 None admin] RBAC: Adding query filter params (group.id=ad10e76c7d0042078bb574adb4da63df, role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=23a2e92a012a4c6995bc96295899d286) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:13.713491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194f963-7342-4091-9b39-e3ff3b98b726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:13.715747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3194f963-7342-4091-9b39-e3ff3b98b726 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:13.728250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41240743-7cf4-48f8-b9e4-4e0537fee0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:13.728250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41240743-7cf4-48f8-b9e4-4e0537fee0ef None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:13.728250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41240743-7cf4-48f8-b9e4-4e0537fee0ef None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:13.728250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41240743-7cf4-48f8-b9e4-4e0537fee0ef None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:13.729010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 222/433] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:13 2022] GET /identity/v3/role_assignments?group.id=ad10e76c7d0042078bb574adb4da63df&scope.project.id=23a2e92a012a4c6995bc96295899d286&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 611 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:13.747788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41240743-7cf4-48f8-b9e4-4e0537fee0ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-41240743-7cf4-48f8-b9e4-4e0537fee0ef None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:13.757614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:13.758583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:13.759370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:13.762253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 212/434] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:13.769029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194f963-7342-4091-9b39-e3ff3b98b726 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:13.861179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:13.881982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194f963-7342-4091-9b39-e3ff3b98b726 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:13.919791 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e695d3-000e-4a81-86f4-822150d4a60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:13.920472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e695d3-000e-4a81-86f4-822150d4a60a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:13.920755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e695d3-000e-4a81-86f4-822150d4a60a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:13.921032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e695d3-000e-4a81-86f4-822150d4a60a None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:13.921942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e695d3-000e-4a81-86f4-822150d4a60a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:13.922250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e695d3-000e-4a81-86f4-822150d4a60a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:13.923631 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e695d3-000e-4a81-86f4-822150d4a60a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:13.932724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 223/435] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:13.938580 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41240743-7cf4-48f8-b9e4-4e0537fee0ef None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:14.007005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:14.049571 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41240743-7cf4-48f8-b9e4-4e0537fee0ef None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:14.085291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac67e5-4878-47ef-a894-f429dd7305bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:14.085973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac67e5-4878-47ef-a894-f429dd7305bb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.086257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac67e5-4878-47ef-a894-f429dd7305bb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.086533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac67e5-4878-47ef-a894-f429dd7305bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.159306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac67e5-4878-47ef-a894-f429dd7305bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:14.160776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dac67e5-4878-47ef-a894-f429dd7305bb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:14.224484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:14.262848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a158c2e-9e1b-4f4e-9f71-f6ce82b4fade None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.263347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a158c2e-9e1b-4f4e-9f71-f6ce82b4fade None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.263454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a158c2e-9e1b-4f4e-9f71-f6ce82b4fade None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.267438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 224/436] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:14:14.273770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40cfd0-a88e-4ae7-aedc-1146bf566e62 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.274193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40cfd0-a88e-4ae7-aedc-1146bf566e62 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.274832 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40cfd0-a88e-4ae7-aedc-1146bf566e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.277808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 213/437] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:14:14.283535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac67e5-4878-47ef-a894-f429dd7305bb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:14.297405 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1b40cfd0-a88e-4ae7-aedc-1146bf566e62 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:14.298945 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1b40cfd0-a88e-4ae7-aedc-1146bf566e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:14.387387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 225/438] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:14.407040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac67e5-4878-47ef-a894-f429dd7305bb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:14.413642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40cfd0-a88e-4ae7-aedc-1146bf566e62 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:14.449459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca821f4-1a97-4e9b-918a-c81ee1f4699d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:14.450335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca821f4-1a97-4e9b-918a-c81ee1f4699d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.450621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca821f4-1a97-4e9b-918a-c81ee1f4699d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.450897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca821f4-1a97-4e9b-918a-c81ee1f4699d None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.452104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca821f4-1a97-4e9b-918a-c81ee1f4699d None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:14.454522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca821f4-1a97-4e9b-918a-c81ee1f4699d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:14.456632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 214/439] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:14 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:14.458053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe69d5-4ae1-45ca-9b52-26c27a7e8d9d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.458413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe69d5-4ae1-45ca-9b52-26c27a7e8d9d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.458639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbe69d5-4ae1-45ca-9b52-26c27a7e8d9d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.460098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 215/440] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:14.461948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe69d5-4ae1-45ca-9b52-26c27a7e8d9d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:14.493962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:14.494252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:14.523400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40cfd0-a88e-4ae7-aedc-1146bf566e62 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:14.559974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf84ff-6ed2-451c-b5a0-a52160c0443e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:14.561020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf84ff-6ed2-451c-b5a0-a52160c0443e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.561299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf84ff-6ed2-451c-b5a0-a52160c0443e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.561606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf84ff-6ed2-451c-b5a0-a52160c0443e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.565226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbe69d5-4ae1-45ca-9b52-26c27a7e8d9d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:14.571537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf84ff-6ed2-451c-b5a0-a52160c0443e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-94cf84ff-6ed2-451c-b5a0-a52160c0443e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:14.579335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:14.580227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:14.581060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 226/441] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:14.581752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df88b8-7d40-4001-a871-dc118744083c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.582172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df88b8-7d40-4001-a871-dc118744083c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.582266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df88b8-7d40-4001-a871-dc118744083c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.600673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f6df88b8-7d40-4001-a871-dc118744083c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:14.601539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e9c01-b997-4a39-b5ca-bdfe039b95b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:14.601918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f6df88b8-7d40-4001-a871-dc118744083c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:14.602395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e9c01-b997-4a39-b5ca-bdfe039b95b6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.602679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e9c01-b997-4a39-b5ca-bdfe039b95b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.602953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e9c01-b997-4a39-b5ca-bdfe039b95b6 None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.604108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e9c01-b997-4a39-b5ca-bdfe039b95b6 None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:14.605563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e9c01-b997-4a39-b5ca-bdfe039b95b6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:14.607155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 216/442] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:14 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:14.607895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e9c01-b997-4a39-b5ca-bdfe039b95b6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:14.674058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 227/443] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:14.675254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df88b8-7d40-4001-a871-dc118744083c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:14.712104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:14.715421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e9c01-b997-4a39-b5ca-bdfe039b95b6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:14.738045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:14.749014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc8434-be28-4809-9a1b-94694e00f5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:14.749751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc8434-be28-4809-9a1b-94694e00f5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.750037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc8434-be28-4809-9a1b-94694e00f5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.750361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cc8434-be28-4809-9a1b-94694e00f5e8 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.751063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc8434-be28-4809-9a1b-94694e00f5e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:14.751366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc8434-be28-4809-9a1b-94694e00f5e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:14.752729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cc8434-be28-4809-9a1b-94694e00f5e8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:14.760806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 217/444] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:14.761621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc8434-be28-4809-9a1b-94694e00f5e8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:14.780985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df88b8-7d40-4001-a871-dc118744083c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:14.814044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bc4f5-2b0a-4ae1-b234-5ce460493d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:14.814686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc4f5-2b0a-4ae1-b234-5ce460493d06 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.814972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc4f5-2b0a-4ae1-b234-5ce460493d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.815252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bc4f5-2b0a-4ae1-b234-5ce460493d06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.816077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bc4f5-2b0a-4ae1-b234-5ce460493d06 None admin] RBAC: Adding query filter params (group.id=ad10e76c7d0042078bb574adb4da63df, role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:14.816385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bc4f5-2b0a-4ae1-b234-5ce460493d06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:14.817851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bc4f5-2b0a-4ae1-b234-5ce460493d06 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:14.823443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 228/445] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:14 2022] GET /identity/v3/role_assignments?group.id=ad10e76c7d0042078bb574adb4da63df&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=3b6f494c47a6407284d1d344940e34cd => generated 611 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:14.824201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bc4f5-2b0a-4ae1-b234-5ce460493d06 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:14.883554 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cc8434-be28-4809-9a1b-94694e00f5e8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:14.925826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff6df0-7b36-489e-9668-2a0aba48fb6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:14.926557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff6df0-7b36-489e-9668-2a0aba48fb6d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.926843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff6df0-7b36-489e-9668-2a0aba48fb6d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.927122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ff6df0-7b36-489e-9668-2a0aba48fb6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.934332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bc4f5-2b0a-4ae1-b234-5ce460493d06 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:14.939751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ff6df0-7b36-489e-9668-2a0aba48fb6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-91ff6df0-7b36-489e-9668-2a0aba48fb6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:14.947716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:14.948690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:14.949556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 218/446] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:14.955152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff6df0-7b36-489e-9668-2a0aba48fb6d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:14.967220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:14.977792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90256cb9-b259-49c2-9f0e-7ac86cf62669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:14.978444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90256cb9-b259-49c2-9f0e-7ac86cf62669 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:14.978731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90256cb9-b259-49c2-9f0e-7ac86cf62669 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:14.979016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90256cb9-b259-49c2-9f0e-7ac86cf62669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:14.983617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:15.058875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90256cb9-b259-49c2-9f0e-7ac86cf62669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:15.060332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90256cb9-b259-49c2-9f0e-7ac86cf62669 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:15.065075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ff6df0-7b36-489e-9668-2a0aba48fb6d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:15.098346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4cb45-1a6f-4d5b-95c8-c1f47abbe92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:15.099005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4cb45-1a6f-4d5b-95c8-c1f47abbe92f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:15.099293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4cb45-1a6f-4d5b-95c8-c1f47abbe92f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:15.099566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff4cb45-1a6f-4d5b-95c8-c1f47abbe92f None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:15.100316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4cb45-1a6f-4d5b-95c8-c1f47abbe92f None admin] RBAC: Adding query filter params (name=member) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:15.100625 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4cb45-1a6f-4d5b-95c8-c1f47abbe92f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:15.102019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff4cb45-1a6f-4d5b-95c8-c1f47abbe92f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:15.110540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 219/447] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:14:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:15.115977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4cb45-1a6f-4d5b-95c8-c1f47abbe92f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:15.157369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 229/448] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:15.162809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90256cb9-b259-49c2-9f0e-7ac86cf62669 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:15.188950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:15.199527 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:15.215158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff4cb45-1a6f-4d5b-95c8-c1f47abbe92f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:15.248690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8ba1e-f77a-4524-825f-8676ca9b92b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:15.249341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8ba1e-f77a-4524-825f-8676ca9b92b4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:15.250202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8ba1e-f77a-4524-825f-8676ca9b92b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:15.250462 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8ba1e-f77a-4524-825f-8676ca9b92b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:15.261579 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90256cb9-b259-49c2-9f0e-7ac86cf62669 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:15.295947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc418e9-4d6d-4b62-aa11-2002635b0735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:15.296912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc418e9-4d6d-4b62-aa11-2002635b0735 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:15.297194 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc418e9-4d6d-4b62-aa11-2002635b0735 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:15.297497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc418e9-4d6d-4b62-aa11-2002635b0735 None admin] PATH_INFO: `/v3/projects/65457e2fc15c4d94825d2ac5ee397f82` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:15.298681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc418e9-4d6d-4b62-aa11-2002635b0735 None admin] RBAC: Authorizing `identity:get_project(project_id=65457e2fc15c4d94825d2ac5ee397f82)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:15.300129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc418e9-4d6d-4b62-aa11-2002635b0735 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:15.301844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 230/449] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:15 2022] GET /identity/v3/projects/65457e2fc15c4d94825d2ac5ee397f82 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:15.307376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc418e9-4d6d-4b62-aa11-2002635b0735 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:15.317604 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8ba1e-f77a-4524-825f-8676ca9b92b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:15.319047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8ba1e-f77a-4524-825f-8676ca9b92b4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:15.398422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:15.400542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:15.405355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc418e9-4d6d-4b62-aa11-2002635b0735 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:15.417054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 220/450] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:14:15.422959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8ba1e-f77a-4524-825f-8676ca9b92b4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:15.440467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d47941-a046-418b-9c13-ec061bb4490b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:15.441256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d47941-a046-418b-9c13-ec061bb4490b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:15.441567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d47941-a046-418b-9c13-ec061bb4490b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:15.441845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d47941-a046-418b-9c13-ec061bb4490b None admin] PATH_INFO: `/v3/groups/67172b0ddcca4d3bbae81117879cfd38` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:15.443018 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d47941-a046-418b-9c13-ec061bb4490b None admin] RBAC: Authorizing `identity:get_group(group_id=67172b0ddcca4d3bbae81117879cfd38)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:15.444470 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d47941-a046-418b-9c13-ec061bb4490b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:15.446107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 231/451] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:15 2022] GET /identity/v3/groups/67172b0ddcca4d3bbae81117879cfd38 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:15.451763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d47941-a046-418b-9c13-ec061bb4490b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:15.519298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8ba1e-f77a-4524-825f-8676ca9b92b4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:15.548911 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d47941-a046-418b-9c13-ec061bb4490b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:15.552042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55eaad-25f9-4a64-8dff-709457d60434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:15.552895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55eaad-25f9-4a64-8dff-709457d60434 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:15.553178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55eaad-25f9-4a64-8dff-709457d60434 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:15.553485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed55eaad-25f9-4a64-8dff-709457d60434 None admin] PATH_INFO: `/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:15.554669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55eaad-25f9-4a64-8dff-709457d60434 None admin] RBAC: Authorizing `identity:get_project(project_id=8faaa2ee2ac54d04b66c0dc66612d84b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:15.556091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed55eaad-25f9-4a64-8dff-709457d60434 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:15.557782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 221/452] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:14:15 2022] GET /identity/v3/projects/8faaa2ee2ac54d04b66c0dc66612d84b => generated 302 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:15.564021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55eaad-25f9-4a64-8dff-709457d60434 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:15.585325 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90501250-cc4d-443b-a8fe-1247f9a602c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:15.586005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90501250-cc4d-443b-a8fe-1247f9a602c3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:15.586295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90501250-cc4d-443b-a8fe-1247f9a602c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:15.586576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90501250-cc4d-443b-a8fe-1247f9a602c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:15.587410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90501250-cc4d-443b-a8fe-1247f9a602c3 None admin] RBAC: Adding query filter params (group.id=67172b0ddcca4d3bbae81117879cfd38, role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=65457e2fc15c4d94825d2ac5ee397f82) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:15.587724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90501250-cc4d-443b-a8fe-1247f9a602c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:15.589168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90501250-cc4d-443b-a8fe-1247f9a602c3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:15.595060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 232/453] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:15 2022] GET /identity/v3/role_assignments?group.id=67172b0ddcca4d3bbae81117879cfd38&scope.project.id=65457e2fc15c4d94825d2ac5ee397f82&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:15.618094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:15.668919 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed55eaad-25f9-4a64-8dff-709457d60434 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:15.705131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436c16b-4242-4448-bc47-ac85b6aa8890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:15.705946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436c16b-4242-4448-bc47-ac85b6aa8890 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:15.706226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436c16b-4242-4448-bc47-ac85b6aa8890 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:15.706499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436c16b-4242-4448-bc47-ac85b6aa8890 None admin] PATH_INFO: `/v3/groups/ad10e76c7d0042078bb574adb4da63df` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:15.707751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436c16b-4242-4448-bc47-ac85b6aa8890 None admin] RBAC: Authorizing `identity:get_group(group_id=ad10e76c7d0042078bb574adb4da63df)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:15.709224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436c16b-4242-4448-bc47-ac85b6aa8890 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:15.710913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 222/454] 10.222.0.23 () {56 vars in 1180 bytes} [Tue Jul 12 13:14:15 2022] GET /identity/v3/groups/ad10e76c7d0042078bb574adb4da63df => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:15.716775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90501250-cc4d-443b-a8fe-1247f9a602c3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:15.745710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:15.820624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90501250-cc4d-443b-a8fe-1247f9a602c3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:15.852721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f93620b-c595-4802-8e20-3c8b6e63f2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:15.853356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93620b-c595-4802-8e20-3c8b6e63f2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:15.853669 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93620b-c595-4802-8e20-3c8b6e63f2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:15.853942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f93620b-c595-4802-8e20-3c8b6e63f2c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:15.854767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93620b-c595-4802-8e20-3c8b6e63f2c3 None admin] RBAC: Adding query filter params (group.id=ad10e76c7d0042078bb574adb4da63df, role.id=5a1c0efbb0f14248bf3a2f523478eef2, scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:15.855075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93620b-c595-4802-8e20-3c8b6e63f2c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:15.856490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f93620b-c595-4802-8e20-3c8b6e63f2c3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:15.862111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 233/455] 10.222.0.23 () {56 vars in 1376 bytes} [Tue Jul 12 13:14:15 2022] GET /identity/v3/role_assignments?group.id=ad10e76c7d0042078bb574adb4da63df&scope.project.id=8faaa2ee2ac54d04b66c0dc66612d84b&role.id=5a1c0efbb0f14248bf3a2f523478eef2 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:18.597860 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a663-e0d9-4251-b5bb-2e825f496442 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:18.598350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a663-e0d9-4251-b5bb-2e825f496442 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:18.598350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b66a663-e0d9-4251-b5bb-2e825f496442 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:18.599934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 223/456] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:18.610535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096140-6ec8-4ed9-9eb1-770f41c4b575 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:18.610938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096140-6ec8-4ed9-9eb1-770f41c4b575 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:18.611103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b096140-6ec8-4ed9-9eb1-770f41c4b575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:18.634377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7b096140-6ec8-4ed9-9eb1-770f41c4b575 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:18.635872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7b096140-6ec8-4ed9-9eb1-770f41c4b575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:18.723691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 234/457] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:18.762416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9226-63e0-49fc-b980-d4949b834185 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:18.762938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9226-63e0-49fc-b980-d4949b834185 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:18.763224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d9226-63e0-49fc-b980-d4949b834185 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:18.764396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096140-6ec8-4ed9-9eb1-770f41c4b575 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:18.764810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 224/458] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:18.770495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee66ae3-b579-499a-a7b3-c9d490fba6c6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:18.770789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee66ae3-b579-499a-a7b3-c9d490fba6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:18.771070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee66ae3-b579-499a-a7b3-c9d490fba6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:18.787864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:18.806406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-dee66ae3-b579-499a-a7b3-c9d490fba6c6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:18.807917 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dee66ae3-b579-499a-a7b3-c9d490fba6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:18.902087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b096140-6ec8-4ed9-9eb1-770f41c4b575 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:18.941848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dca64-4810-45f7-968a-6959fa436377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:18.942998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dca64-4810-45f7-968a-6959fa436377 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:18.944247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dca64-4810-45f7-968a-6959fa436377 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:18.945390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dca64-4810-45f7-968a-6959fa436377 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:18.961313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dca64-4810-45f7-968a-6959fa436377 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:18.962830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dca64-4810-45f7-968a-6959fa436377 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:18.963821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 225/459] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:18.965974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad37a43-bd88-4408-a87a-43070786c392 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:18.966499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad37a43-bd88-4408-a87a-43070786c392 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:18.966975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad37a43-bd88-4408-a87a-43070786c392 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:18.969344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 226/460] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:18.981667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffabdd-ad19-4c20-bb33-da17cb0766d9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:18.982242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffabdd-ad19-4c20-bb33-da17cb0766d9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:18.982748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffabdd-ad19-4c20-bb33-da17cb0766d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-097dca64-4810-45f7-968a-6959fa436377 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:18.997564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:18.998517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:18.999385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:18.999385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:18.999385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:18.999385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:18.999385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 235/461] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:19.000972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dca64-4810-45f7-968a-6959fa436377 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:19.011674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d7ffabdd-ad19-4c20-bb33-da17cb0766d9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:19.012886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d7ffabdd-ad19-4c20-bb33-da17cb0766d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:19.061740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:19.077872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:19.084230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 227/462] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 12 13:14:19.085204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffabdd-ad19-4c20-bb33-da17cb0766d9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:19.123655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dca64-4810-45f7-968a-6959fa436377 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:19.160114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e8707-67db-42ca-9586-9ec1f5cd4367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:19.160928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e8707-67db-42ca-9586-9ec1f5cd4367 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.161309 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e8707-67db-42ca-9586-9ec1f5cd4367 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.161619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e8707-67db-42ca-9586-9ec1f5cd4367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.172878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e8707-67db-42ca-9586-9ec1f5cd4367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:19.174466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e8707-67db-42ca-9586-9ec1f5cd4367 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:19.192885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffabdd-ad19-4c20-bb33-da17cb0766d9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a86e8707-67db-42ca-9586-9ec1f5cd4367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:19.193873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:19.194739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:19.195596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:19.195596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:19.195596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:19.195596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:19.195596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 236/463] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:19.201139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e8707-67db-42ca-9586-9ec1f5cd4367 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:19.229230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a66d1-df95-429e-a9dd-2c9c42d05367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:19.229938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a66d1-df95-429e-a9dd-2c9c42d05367 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.230224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a66d1-df95-429e-a9dd-2c9c42d05367 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.230500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138a66d1-df95-429e-a9dd-2c9c42d05367 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.231350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a66d1-df95-429e-a9dd-2c9c42d05367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:19.231689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a66d1-df95-429e-a9dd-2c9c42d05367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:19.233122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138a66d1-df95-429e-a9dd-2c9c42d05367 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:19.253448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 228/464] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:19.254380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a66d1-df95-429e-a9dd-2c9c42d05367 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:19.299335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:19.317992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e8707-67db-42ca-9586-9ec1f5cd4367 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:19.345247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:19.357874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba1411-9b35-4810-bfcc-69d116a67f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:19.358888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba1411-9b35-4810-bfcc-69d116a67f83 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.359881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba1411-9b35-4810-bfcc-69d116a67f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.360482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eba1411-9b35-4810-bfcc-69d116a67f83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.372006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba1411-9b35-4810-bfcc-69d116a67f83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:19.373758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eba1411-9b35-4810-bfcc-69d116a67f83 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:19.379321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138a66d1-df95-429e-a9dd-2c9c42d05367 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2eba1411-9b35-4810-bfcc-69d116a67f83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:19.394420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:19.395492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:19.396428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:19.396428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:19.396428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:19.396428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:19.402971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 237/465] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:19.404159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba1411-9b35-4810-bfcc-69d116a67f83 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:19.419663 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab7ca5-337c-4a10-aecb-96ef1ce29422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:19.422752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab7ca5-337c-4a10-aecb-96ef1ce29422 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.423282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab7ca5-337c-4a10-aecb-96ef1ce29422 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.423721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ab7ca5-337c-4a10-aecb-96ef1ce29422 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.424606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab7ca5-337c-4a10-aecb-96ef1ce29422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:19.425063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab7ca5-337c-4a10-aecb-96ef1ce29422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:19.426732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ab7ca5-337c-4a10-aecb-96ef1ce29422 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:19.444441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 229/466] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:19.445568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab7ca5-337c-4a10-aecb-96ef1ce29422 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:19.524634 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eba1411-9b35-4810-bfcc-69d116a67f83 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:19.557703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:19.561034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ab7ca5-337c-4a10-aecb-96ef1ce29422 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:19.562328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1211c08-c3c5-4899-b5f0-2c888f5835c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:19.563085 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1211c08-c3c5-4899-b5f0-2c888f5835c7 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.563376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1211c08-c3c5-4899-b5f0-2c888f5835c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.563657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1211c08-c3c5-4899-b5f0-2c888f5835c7 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.564370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1211c08-c3c5-4899-b5f0-2c888f5835c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:19.565843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1211c08-c3c5-4899-b5f0-2c888f5835c7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:19.588015 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 238/467] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:14:19 2022] POST /identity/v3/users => generated 246 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:19.589162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1211c08-c3c5-4899-b5f0-2c888f5835c7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:19.602526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0d196-2f7c-4c38-a558-3bec49bba212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:19.603861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0d196-2f7c-4c38-a558-3bec49bba212 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.604358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0d196-2f7c-4c38-a558-3bec49bba212 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.604818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0d196-2f7c-4c38-a558-3bec49bba212 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.605814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0d196-2f7c-4c38-a558-3bec49bba212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:19.606338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0d196-2f7c-4c38-a558-3bec49bba212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:19.607961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0d196-2f7c-4c38-a558-3bec49bba212 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:19.623284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 230/468] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:19.626172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f423-8209-4138-92b7-74277129c8c6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.626674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f423-8209-4138-92b7-74277129c8c6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.627216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f423-8209-4138-92b7-74277129c8c6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.633453 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 231/469] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:19.634696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6f423-8209-4138-92b7-74277129c8c6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:19.651269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:19.740941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1211c08-c3c5-4899-b5f0-2c888f5835c7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:19.773907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6f423-8209-4138-92b7-74277129c8c6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:19.790490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5c7e1-6dfe-4f96-882c-157ec0a9de1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:19.790786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5c7e1-6dfe-4f96-882c-157ec0a9de1d None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.791088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5c7e1-6dfe-4f96-882c-157ec0a9de1d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.791349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5c7e1-6dfe-4f96-882c-157ec0a9de1d None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.792060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5c7e1-6dfe-4f96-882c-157ec0a9de1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:19.793541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee5c7e1-6dfe-4f96-882c-157ec0a9de1d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:19.818103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 239/470] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:14:19 2022] POST /identity/v3/users => generated 248 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:19.818103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3507ed-f733-4438-ba6b-1d727f43e434 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.818423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3507ed-f733-4438-ba6b-1d727f43e434 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.818708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3507ed-f733-4438-ba6b-1d727f43e434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.822792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc39cb9-c4c4-46ab-8948-cc1dd8cff4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:19.823905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc39cb9-c4c4-46ab-8948-cc1dd8cff4b1 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.824398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc39cb9-c4c4-46ab-8948-cc1dd8cff4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.824876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc39cb9-c4c4-46ab-8948-cc1dd8cff4b1 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.825865 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc39cb9-c4c4-46ab-8948-cc1dd8cff4b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:19.827580 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc39cb9-c4c4-46ab-8948-cc1dd8cff4b1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:19.844681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dd3507ed-f733-4438-ba6b-1d727f43e434 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:19.845970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dd3507ed-f733-4438-ba6b-1d727f43e434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:19.857851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 232/471] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:14:19 2022] POST /identity/v3/users => generated 248 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:19.942194 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 240/472] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:19.976430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24180c43-3b9a-41ce-9165-966ffec7fa22 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:19.977100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24180c43-3b9a-41ce-9165-966ffec7fa22 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:19.977674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24180c43-3b9a-41ce-9165-966ffec7fa22 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:19.979721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 233/473] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:19.981950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3507ed-f733-4438-ba6b-1d727f43e434 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:19.999719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7c691-2b9e-4e57-8aff-b32e3cde803f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:20.001225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7c691-2b9e-4e57-8aff-b32e3cde803f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:20.001754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa7c691-2b9e-4e57-8aff-b32e3cde803f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:20.013444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:20.032027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-daa7c691-2b9e-4e57-8aff-b32e3cde803f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:20.033264 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-daa7c691-2b9e-4e57-8aff-b32e3cde803f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:20.049829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:20.122517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 234/474] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:20.130964 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3507ed-f733-4438-ba6b-1d727f43e434 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:20.145965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7c691-2b9e-4e57-8aff-b32e3cde803f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:20.177962 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67914ad7-59c3-40ff-bd1f-b2a67da9e710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:20.179092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67914ad7-59c3-40ff-bd1f-b2a67da9e710 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:20.179553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67914ad7-59c3-40ff-bd1f-b2a67da9e710 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:20.180010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67914ad7-59c3-40ff-bd1f-b2a67da9e710 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:20.194360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67914ad7-59c3-40ff-bd1f-b2a67da9e710 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:20.196248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67914ad7-59c3-40ff-bd1f-b2a67da9e710 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-67914ad7-59c3-40ff-bd1f-b2a67da9e710 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:20.220946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:20.221920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:20.222786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:20.222786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:20.222786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:20.222786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:20.224888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 241/475] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:20.231185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67914ad7-59c3-40ff-bd1f-b2a67da9e710 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:20.278962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa7c691-2b9e-4e57-8aff-b32e3cde803f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:20.331146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:20.345046 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:20.345753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:20.346040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:20.346314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:20.347371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:20.349177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:20.356801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.sql.core [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 13:14:20.356801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 13:14:20.356801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [parameters: {'id': '26cef659f1774f528d4d88d064e67d1a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 13:14:20.356801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80288) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application result.read() Jul 12 13:14:20.376576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return ref.to_dict() Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:14:20.377687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application yield resource Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise self.value Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application session.commit() Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 13:14:20.378898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.session.flush() Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application compat.raise_( Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 13:14:20.380176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:14:20.381546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application result.read() Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:14:20.382754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [parameters: {'id': '26cef659f1774f528d4d88d064e67d1a', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:20.383590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 12 13:14:20.384378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:20.387387 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 235/476] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 12 13:14:20 2022] POST /identity/v3/roles => generated 144 bytes in 242 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:20.394800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:20.404689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67914ad7-59c3-40ff-bd1f-b2a67da9e710 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:20.412316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:20.478704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26a02a-6a16-4451-acf3-7a8424e692e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:20.479382 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26a02a-6a16-4451-acf3-7a8424e692e4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:20.480041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26a02a-6a16-4451-acf3-7a8424e692e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:20.480326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26a02a-6a16-4451-acf3-7a8424e692e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:20.481069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26a02a-6a16-4451-acf3-7a8424e692e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:20.481394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26a02a-6a16-4451-acf3-7a8424e692e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:20.482799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26a02a-6a16-4451-acf3-7a8424e692e4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:20.509780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 242/477] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:20.518566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26a02a-6a16-4451-acf3-7a8424e692e4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:20.567481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136be0a-17fb-49d1-8f60-6358953f078e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:20.646688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70afbe-85cb-417e-9cd7-8638d9fa325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:20.647579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70afbe-85cb-417e-9cd7-8638d9fa325e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:20.647875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70afbe-85cb-417e-9cd7-8638d9fa325e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:20.648149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b70afbe-85cb-417e-9cd7-8638d9fa325e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:20.657283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26a02a-6a16-4451-acf3-7a8424e692e4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:20.678635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b70afbe-85cb-417e-9cd7-8638d9fa325e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:20.688796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9b70afbe-85cb-417e-9cd7-8638d9fa325e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:20.693602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:20.694679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 12 13:14:20.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:20.698244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b9a8b-a17c-4704-94e1-fc467d74bc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:20.699054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b9a8b-a17c-4704-94e1-fc467d74bc98 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:20.699584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b9a8b-a17c-4704-94e1-fc467d74bc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:20.699773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 236/478] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:20.700134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b9a8b-a17c-4704-94e1-fc467d74bc98 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:20.700659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70afbe-85cb-417e-9cd7-8638d9fa325e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:20.701199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b9a8b-a17c-4704-94e1-fc467d74bc98 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:20.704099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b9a8b-a17c-4704-94e1-fc467d74bc98 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:20.726103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 243/479] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:14:20 2022] POST /identity/v3/users => generated 249 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:20.832750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:20.875332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b70afbe-85cb-417e-9cd7-8638d9fa325e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:20.916232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d43c25-1ad2-42a3-81c6-44deda535ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:20.916904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d43c25-1ad2-42a3-81c6-44deda535ba9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:20.917197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d43c25-1ad2-42a3-81c6-44deda535ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:20.917509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d43c25-1ad2-42a3-81c6-44deda535ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:20.918275 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d43c25-1ad2-42a3-81c6-44deda535ba9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:20.918639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d43c25-1ad2-42a3-81c6-44deda535ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:20.920022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d43c25-1ad2-42a3-81c6-44deda535ba9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:20.929555 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 237/480] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:22.059155 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406381-3926-489d-aa00-44bac1071e03 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.059670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406381-3926-489d-aa00-44bac1071e03 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.059744 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406381-3926-489d-aa00-44bac1071e03 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.061318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 244/481] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:22.067411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b791b-c2c3-40ff-9b2d-d44145e334be None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.067820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b791b-c2c3-40ff-9b2d-d44145e334be None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.067986 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b791b-c2c3-40ff-9b2d-d44145e334be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.085122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ce3b791b-c2c3-40ff-9b2d-d44145e334be None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:22.086440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ce3b791b-c2c3-40ff-9b2d-d44145e334be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:22.176025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 238/482] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:22.222843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d406381-3926-489d-aa00-44bac1071e03 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:22.371099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d406381-3926-489d-aa00-44bac1071e03 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:22.422501 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a091b7-d5a5-4eb7-909d-6467a28ef553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:22.423317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a091b7-d5a5-4eb7-909d-6467a28ef553 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.423608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a091b7-d5a5-4eb7-909d-6467a28ef553 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.423894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a091b7-d5a5-4eb7-909d-6467a28ef553 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.429574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8969388-3737-4587-bb83-1e1f05520acc None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.429957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8969388-3737-4587-bb83-1e1f05520acc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.430236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8969388-3737-4587-bb83-1e1f05520acc None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.431854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 239/483] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:14:22.437585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec0c3d-85de-4e30-ad8f-f3b5697c411c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.437943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec0c3d-85de-4e30-ad8f-f3b5697c411c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.438263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec0c3d-85de-4e30-ad8f-f3b5697c411c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.450457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a091b7-d5a5-4eb7-909d-6467a28ef553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:22.458137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-48ec0c3d-85de-4e30-ad8f-f3b5697c411c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:22.459578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-48ec0c3d-85de-4e30-ad8f-f3b5697c411c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b7a091b7-d5a5-4eb7-909d-6467a28ef553 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:22.459684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:22.460487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:22.461258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:22.471450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 245/484] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:22.472569 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a091b7-d5a5-4eb7-909d-6467a28ef553 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:22.547655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:22.550755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:22.575892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 240/485] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:22.631293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a091b7-d5a5-4eb7-909d-6467a28ef553 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:22.641097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec0c3d-85de-4e30-ad8f-f3b5697c411c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:22.668460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19793af0-1e1e-4fdc-a440-fe7fe6a6ea3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:22.669418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19793af0-1e1e-4fdc-a440-fe7fe6a6ea3c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.669875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19793af0-1e1e-4fdc-a440-fe7fe6a6ea3c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.670321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19793af0-1e1e-4fdc-a440-fe7fe6a6ea3c None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.671236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19793af0-1e1e-4fdc-a440-fe7fe6a6ea3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:22.671769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19793af0-1e1e-4fdc-a440-fe7fe6a6ea3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:22.673306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19793af0-1e1e-4fdc-a440-fe7fe6a6ea3c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:22.683075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 246/486] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:22.689840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19793af0-1e1e-4fdc-a440-fe7fe6a6ea3c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:22.802442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec0c3d-85de-4e30-ad8f-f3b5697c411c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:22.835556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19793af0-1e1e-4fdc-a440-fe7fe6a6ea3c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:22.845151 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84287259-a78e-4754-b6f5-4aac0be95f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:22.846198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84287259-a78e-4754-b6f5-4aac0be95f52 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.846654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84287259-a78e-4754-b6f5-4aac0be95f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.847475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:22.848523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84287259-a78e-4754-b6f5-4aac0be95f52 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.863328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84287259-a78e-4754-b6f5-4aac0be95f52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-84287259-a78e-4754-b6f5-4aac0be95f52 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:22.875298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:22.876179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:22.876957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:22.878889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 241/487] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:22.879762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a1e88-dcbc-400b-8f7f-4d4ed7bae7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:22.880549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a1e88-dcbc-400b-8f7f-4d4ed7bae7a7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.880835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a1e88-dcbc-400b-8f7f-4d4ed7bae7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.881112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a1e88-dcbc-400b-8f7f-4d4ed7bae7a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.882746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26bddc-6ebf-4414-a030-89e6f9558d84 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.883568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26bddc-6ebf-4414-a030-89e6f9558d84 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.884505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26bddc-6ebf-4414-a030-89e6f9558d84 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.886594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 242/488] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:22.887617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26bddc-6ebf-4414-a030-89e6f9558d84 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:22.898267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a1e88-dcbc-400b-8f7f-4d4ed7bae7a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:22.900200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a1e88-dcbc-400b-8f7f-4d4ed7bae7a7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b72a1e88-dcbc-400b-8f7f-4d4ed7bae7a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:22.920108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:22.920975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:22.922556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:22.922556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:22.922556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:22.922556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:22.927485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 247/489] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:22.927485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724bb5-649a-446d-8bb2-222f96675338 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:22.927485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724bb5-649a-446d-8bb2-222f96675338 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:22.927485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa724bb5-649a-446d-8bb2-222f96675338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:22.945021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-aa724bb5-649a-446d-8bb2-222f96675338 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:22.946280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-aa724bb5-649a-446d-8bb2-222f96675338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:22.965707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:23.021726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26bddc-6ebf-4414-a030-89e6f9558d84 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.025635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 248/490] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:23.026536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724bb5-649a-446d-8bb2-222f96675338 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.058618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacc36e-9da8-4e63-9736-840e9c0e409a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.059660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacc36e-9da8-4e63-9736-840e9c0e409a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.060169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacc36e-9da8-4e63-9736-840e9c0e409a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.060635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eacc36e-9da8-4e63-9736-840e9c0e409a None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.061920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eacc36e-9da8-4e63-9736-840e9c0e409a None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:23.062498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eacc36e-9da8-4e63-9736-840e9c0e409a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:23.064185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eacc36e-9da8-4e63-9736-840e9c0e409a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:23.073447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 243/491] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:23.074971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacc36e-9da8-4e63-9736-840e9c0e409a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.147123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:23.158989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa724bb5-649a-446d-8bb2-222f96675338 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.180910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eacc36e-9da8-4e63-9736-840e9c0e409a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.196902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:23.197073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd89d1b-35c1-4d3f-8a8e-d92c8b40db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.197935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd89d1b-35c1-4d3f-8a8e-d92c8b40db75 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.198041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd89d1b-35c1-4d3f-8a8e-d92c8b40db75 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.198619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd89d1b-35c1-4d3f-8a8e-d92c8b40db75 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.199062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd89d1b-35c1-4d3f-8a8e-d92c8b40db75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:23.199375 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd89d1b-35c1-4d3f-8a8e-d92c8b40db75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:23.200750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd89d1b-35c1-4d3f-8a8e-d92c8b40db75 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:23.219495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 249/492] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:23.219495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd89d1b-35c1-4d3f-8a8e-d92c8b40db75 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.225406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cfc37-e022-42aa-989b-7fc0b95a3114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.226460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cfc37-e022-42aa-989b-7fc0b95a3114 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.226937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cfc37-e022-42aa-989b-7fc0b95a3114 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.227405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cfc37-e022-42aa-989b-7fc0b95a3114 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.239949 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cfc37-e022-42aa-989b-7fc0b95a3114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-af9cfc37-e022-42aa-989b-7fc0b95a3114 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:23.248269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:23.249189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:23.251090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:23.253841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 244/493] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:23.254959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cfc37-e022-42aa-989b-7fc0b95a3114 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.350210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd89d1b-35c1-4d3f-8a8e-d92c8b40db75 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.360536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cfc37-e022-42aa-989b-7fc0b95a3114 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.388871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a6196-07bb-46aa-ae0d-fd70ae393751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.389712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a6196-07bb-46aa-ae0d-fd70ae393751 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.390002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a6196-07bb-46aa-ae0d-fd70ae393751 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.390411 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a6196-07bb-46aa-ae0d-fd70ae393751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.396161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2bf0d0-fa47-4def-b962-6163843c08db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.396968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bf0d0-fa47-4def-b962-6163843c08db None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.397302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bf0d0-fa47-4def-b962-6163843c08db None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.397555 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2bf0d0-fa47-4def-b962-6163843c08db None admin] PATH_INFO: `/v3/users/nova` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.402157 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a6196-07bb-46aa-ae0d-fd70ae393751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:23.403616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a6196-07bb-46aa-ae0d-fd70ae393751 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:23.405335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bf0d0-fa47-4def-b962-6163843c08db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:23.407542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2bf0d0-fa47-4def-b962-6163843c08db None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3f2bf0d0-fa47-4def-b962-6163843c08db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:23.424019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:23.424965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:23.431064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-df1a6196-07bb-46aa-ae0d-fd70ae393751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:23.431587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:23.432312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:23.437656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:23.437656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:23.437656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:23.437656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:23.437656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 250/494] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:23.437656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a6196-07bb-46aa-ae0d-fd70ae393751 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.438245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 245/495] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:23.438245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba194e57-7d60-468a-adc6-f2772d46b058 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.438245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba194e57-7d60-468a-adc6-f2772d46b058 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.438245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba194e57-7d60-468a-adc6-f2772d46b058 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.438245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 246/496] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:23.438245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba194e57-7d60-468a-adc6-f2772d46b058 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.438658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:23.479171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:23.540460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a6196-07bb-46aa-ae0d-fd70ae393751 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.552201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba194e57-7d60-468a-adc6-f2772d46b058 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.576816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29309646-08c1-4be7-ba26-834f22f6e71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.577516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29309646-08c1-4be7-ba26-834f22f6e71d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.577899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29309646-08c1-4be7-ba26-834f22f6e71d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.578168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29309646-08c1-4be7-ba26-834f22f6e71d None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.579092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29309646-08c1-4be7-ba26-834f22f6e71d None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:23.579400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29309646-08c1-4be7-ba26-834f22f6e71d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:23.580775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29309646-08c1-4be7-ba26-834f22f6e71d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:23.585491 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059040d-8cfe-4fbe-b2ea-a53dd11f6e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.586177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059040d-8cfe-4fbe-b2ea-a53dd11f6e12 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.586447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059040d-8cfe-4fbe-b2ea-a53dd11f6e12 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.586728 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059040d-8cfe-4fbe-b2ea-a53dd11f6e12 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.588101 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059040d-8cfe-4fbe-b2ea-a53dd11f6e12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:23.588664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059040d-8cfe-4fbe-b2ea-a53dd11f6e12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:23.588885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 251/497] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:23.589925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29309646-08c1-4be7-ba26-834f22f6e71d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.590313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059040d-8cfe-4fbe-b2ea-a53dd11f6e12 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:23.605245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 247/498] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:23.606563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbac2f8-c9b9-4520-a521-f02cdefce6b6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.606945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbac2f8-c9b9-4520-a521-f02cdefce6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.607238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbac2f8-c9b9-4520-a521-f02cdefce6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.624172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1bbac2f8-c9b9-4520-a521-f02cdefce6b6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:23.625609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1bbac2f8-c9b9-4520-a521-f02cdefce6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:23.653778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:23.692165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29309646-08c1-4be7-ba26-834f22f6e71d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.693469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 248/499] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:23.694602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb4eb-a8ca-4624-8cff-e3f94e975ae5 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.694893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb4eb-a8ca-4624-8cff-e3f94e975ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.695178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb4eb-a8ca-4624-8cff-e3f94e975ae5 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.696618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 249/500] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:23.698130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb4eb-a8ca-4624-8cff-e3f94e975ae5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.725545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:23.727696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f63894-1296-4503-a5af-357639f6f015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.728513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f63894-1296-4503-a5af-357639f6f015 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.728828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f63894-1296-4503-a5af-357639f6f015 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.729107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f63894-1296-4503-a5af-357639f6f015 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.729982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f63894-1296-4503-a5af-357639f6f015 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:23.730292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f63894-1296-4503-a5af-357639f6f015 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:23.731717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f63894-1296-4503-a5af-357639f6f015 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:23.767653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 252/501] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:23.768464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f63894-1296-4503-a5af-357639f6f015 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.803571 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb4eb-a8ca-4624-8cff-e3f94e975ae5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.836797 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa6dcd1-d12e-4f5f-93a6-51ea65dd213d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.837612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6dcd1-d12e-4f5f-93a6-51ea65dd213d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.837898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6dcd1-d12e-4f5f-93a6-51ea65dd213d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.838172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa6dcd1-d12e-4f5f-93a6-51ea65dd213d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.848558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6dcd1-d12e-4f5f-93a6-51ea65dd213d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:23.850038 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa6dcd1-d12e-4f5f-93a6-51ea65dd213d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-bfa6dcd1-d12e-4f5f-93a6-51ea65dd213d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:23.867081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:23.867885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:23.868720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:23.868720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:23.868720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:23.868720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:23.868720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 250/502] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:23.869469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71ac0a-1f80-45ee-becb-0a7b271b4dbe None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.869713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71ac0a-1f80-45ee-becb-0a7b271b4dbe None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.869986 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d71ac0a-1f80-45ee-becb-0a7b271b4dbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.875065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f63894-1296-4503-a5af-357639f6f015 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:23.885939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3d71ac0a-1f80-45ee-becb-0a7b271b4dbe None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:23.887307 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3d71ac0a-1f80-45ee-becb-0a7b271b4dbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:23.900230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:23.912929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e249e88-07df-4ae7-bcbf-49afdb310345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:23.913688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e249e88-07df-4ae7-bcbf-49afdb310345 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:23.913973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e249e88-07df-4ae7-bcbf-49afdb310345 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:23.914247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e249e88-07df-4ae7-bcbf-49afdb310345 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:23.921168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e249e88-07df-4ae7-bcbf-49afdb310345 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:23.923292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e249e88-07df-4ae7-bcbf-49afdb310345 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8e249e88-07df-4ae7-bcbf-49afdb310345 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:23.939614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:23.940604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 253/503] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:23.941595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e249e88-07df-4ae7-bcbf-49afdb310345 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.959495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 251/504] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:23.960334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71ac0a-1f80-45ee-becb-0a7b271b4dbe None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:23.968170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:24.043650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e249e88-07df-4ae7-bcbf-49afdb310345 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.059468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d71ac0a-1f80-45ee-becb-0a7b271b4dbe None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.077963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6058d-3eb5-4c61-ba50-151222e5ab2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.078749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6058d-3eb5-4c61-ba50-151222e5ab2c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.079035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6058d-3eb5-4c61-ba50-151222e5ab2c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.079310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d6058d-3eb5-4c61-ba50-151222e5ab2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.089104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6058d-3eb5-4c61-ba50-151222e5ab2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:24.090568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d6058d-3eb5-4c61-ba50-151222e5ab2c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:24.092325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ab0f7-9e3b-4c60-a90f-a77251cc45da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.092973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ab0f7-9e3b-4c60-a90f-a77251cc45da None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.093775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ab0f7-9e3b-4c60-a90f-a77251cc45da None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.094075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ab0f7-9e3b-4c60-a90f-a77251cc45da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-11d6058d-3eb5-4c61-ba50-151222e5ab2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:24.108712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:24.109795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:24.110626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:24.110626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:24.110626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:24.110626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:24.110626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 254/505] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:24.110888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6058d-3eb5-4c61-ba50-151222e5ab2c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.145061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:24.164733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ab0f7-9e3b-4c60-a90f-a77251cc45da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:24.166186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ab0f7-9e3b-4c60-a90f-a77251cc45da None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:24.175035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:24.210000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d6058d-3eb5-4c61-ba50-151222e5ab2c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.245524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955f06d-c4d9-48ec-8977-7372b6bde72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.246426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955f06d-c4d9-48ec-8977-7372b6bde72d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.246715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955f06d-c4d9-48ec-8977-7372b6bde72d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.246995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955f06d-c4d9-48ec-8977-7372b6bde72d None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.247730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955f06d-c4d9-48ec-8977-7372b6bde72d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:24.248042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955f06d-c4d9-48ec-8977-7372b6bde72d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:24.249446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955f06d-c4d9-48ec-8977-7372b6bde72d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:24.261081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 255/506] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:24.261956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955f06d-c4d9-48ec-8977-7372b6bde72d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.266781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 252/507] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:24.267788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ab0f7-9e3b-4c60-a90f-a77251cc45da None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.354742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:24.362082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955f06d-c4d9-48ec-8977-7372b6bde72d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.365163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ab0f7-9e3b-4c60-a90f-a77251cc45da None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.372245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:24.395454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69543d9-70e2-469f-a07c-b54bd4f72022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.396125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69543d9-70e2-469f-a07c-b54bd4f72022 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.396409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69543d9-70e2-469f-a07c-b54bd4f72022 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.396686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69543d9-70e2-469f-a07c-b54bd4f72022 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.397564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69543d9-70e2-469f-a07c-b54bd4f72022 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:24.397875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69543d9-70e2-469f-a07c-b54bd4f72022 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:24.398812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905b5fd-5271-4625-8576-98a4d09cb613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.399323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69543d9-70e2-469f-a07c-b54bd4f72022 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:24.399651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905b5fd-5271-4625-8576-98a4d09cb613 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.399939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905b5fd-5271-4625-8576-98a4d09cb613 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.400214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905b5fd-5271-4625-8576-98a4d09cb613 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.410797 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905b5fd-5271-4625-8576-98a4d09cb613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:24.416605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 256/508] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 12 13:14:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:24.417334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69543d9-70e2-469f-a07c-b54bd4f72022 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-c905b5fd-5271-4625-8576-98a4d09cb613 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:24.418566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:24.419436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:24.420232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 253/509] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:24.422270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905b5fd-5271-4625-8576-98a4d09cb613 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.517334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69543d9-70e2-469f-a07c-b54bd4f72022 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.518514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905b5fd-5271-4625-8576-98a4d09cb613 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.551872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6873782-510b-4f28-bb57-437e395df580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.552658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6873782-510b-4f28-bb57-437e395df580 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.552940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6873782-510b-4f28-bb57-437e395df580 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.553215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6873782-510b-4f28-bb57-437e395df580 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.553343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c290ef-c74f-477b-8fc8-01290e71bfd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.554091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c290ef-c74f-477b-8fc8-01290e71bfd0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.554440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c290ef-c74f-477b-8fc8-01290e71bfd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.554799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c290ef-c74f-477b-8fc8-01290e71bfd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.555554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c290ef-c74f-477b-8fc8-01290e71bfd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:24.555864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c290ef-c74f-477b-8fc8-01290e71bfd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:24.557239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c290ef-c74f-477b-8fc8-01290e71bfd0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:24.560556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6873782-510b-4f28-bb57-437e395df580 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:24.562812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6873782-510b-4f28-bb57-437e395df580 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:24.569173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 257/510] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:24.570001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c290ef-c74f-477b-8fc8-01290e71bfd0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f6873782-510b-4f28-bb57-437e395df580 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:24.578679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:24.579557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:24.580442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 254/511] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:24 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:24.581570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6873782-510b-4f28-bb57-437e395df580 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.585011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:24.609712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:24.670601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c290ef-c74f-477b-8fc8-01290e71bfd0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.681181 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6873782-510b-4f28-bb57-437e395df580 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.703765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904006a1-a6f8-4da8-ad07-342d33883d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.704553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904006a1-a6f8-4da8-ad07-342d33883d10 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.704833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904006a1-a6f8-4da8-ad07-342d33883d10 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.705101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904006a1-a6f8-4da8-ad07-342d33883d10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.714551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe01d2b-ad1d-495d-8da8-f3370819846d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.715189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe01d2b-ad1d-495d-8da8-f3370819846d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.715340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904006a1-a6f8-4da8-ad07-342d33883d10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:24.715474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe01d2b-ad1d-495d-8da8-f3370819846d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.715749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe01d2b-ad1d-495d-8da8-f3370819846d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.716977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904006a1-a6f8-4da8-ad07-342d33883d10 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-904006a1-a6f8-4da8-ad07-342d33883d10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:24.735612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:24.736673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:24.742429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:24.742429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:24.742429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:24.742429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:24.742429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 258/512] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:24.742429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904006a1-a6f8-4da8-ad07-342d33883d10 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.783355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe01d2b-ad1d-495d-8da8-f3370819846d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:24.784802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe01d2b-ad1d-495d-8da8-f3370819846d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:24.800768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:24.811885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:24.840905 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904006a1-a6f8-4da8-ad07-342d33883d10 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.873573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c5b9-8497-4b58-926c-2a66a6b8e0b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:24.874232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c5b9-8497-4b58-926c-2a66a6b8e0b6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:24.874585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c5b9-8497-4b58-926c-2a66a6b8e0b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:24.874932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c5b9-8497-4b58-926c-2a66a6b8e0b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:24.875724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63c5b9-8497-4b58-926c-2a66a6b8e0b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:24.876031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63c5b9-8497-4b58-926c-2a66a6b8e0b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:24.877416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63c5b9-8497-4b58-926c-2a66a6b8e0b6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:24.883772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 255/513] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:24.884584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe01d2b-ad1d-495d-8da8-f3370819846d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.885251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 259/514] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:24.886366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c5b9-8497-4b58-926c-2a66a6b8e0b6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:24.985856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c5b9-8497-4b58-926c-2a66a6b8e0b6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:24.988362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe01d2b-ad1d-495d-8da8-f3370819846d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.011268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:25.014867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:25.019958 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5e1fc-f1d3-406a-a3dd-6f53d54e8219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.020631 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5e1fc-f1d3-406a-a3dd-6f53d54e8219 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.020916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5e1fc-f1d3-406a-a3dd-6f53d54e8219 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.021195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c5e1fc-f1d3-406a-a3dd-6f53d54e8219 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.022065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5e1fc-f1d3-406a-a3dd-6f53d54e8219 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:25.022376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5e1fc-f1d3-406a-a3dd-6f53d54e8219 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.023561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07214b5-a467-46fc-a1a2-35a5d0a86e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.023803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c5e1fc-f1d3-406a-a3dd-6f53d54e8219 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.024230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07214b5-a467-46fc-a1a2-35a5d0a86e8e None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.024517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07214b5-a467-46fc-a1a2-35a5d0a86e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.024796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07214b5-a467-46fc-a1a2-35a5d0a86e8e None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.025531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07214b5-a467-46fc-a1a2-35a5d0a86e8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.026962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07214b5-a467-46fc-a1a2-35a5d0a86e8e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.041494 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 260/515] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 12 13:14:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:25.042295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5e1fc-f1d3-406a-a3dd-6f53d54e8219 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.064371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 256/516] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:14:24 2022] POST /identity/v3/users => generated 247 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:25.064371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07214b5-a467-46fc-a1a2-35a5d0a86e8e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.156004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c5e1fc-f1d3-406a-a3dd-6f53d54e8219 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.179369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07214b5-a467-46fc-a1a2-35a5d0a86e8e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.196366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9487-fb4f-40ef-8bae-b73753738a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.196890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9487-fb4f-40ef-8bae-b73753738a35 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.197175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9487-fb4f-40ef-8bae-b73753738a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.197477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b9487-fb4f-40ef-8bae-b73753738a35 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.198209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b9487-fb4f-40ef-8bae-b73753738a35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:25.198515 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b9487-fb4f-40ef-8bae-b73753738a35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.200144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1b9487-fb4f-40ef-8bae-b73753738a35 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.212679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 261/517] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:25.213530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9487-fb4f-40ef-8bae-b73753738a35 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.223940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbf83c-b6f8-4485-bcdc-07dd26f2a9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.225190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbf83c-b6f8-4485-bcdc-07dd26f2a9a3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.225558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbf83c-b6f8-4485-bcdc-07dd26f2a9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.226123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddbf83c-b6f8-4485-bcdc-07dd26f2a9a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.240575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbf83c-b6f8-4485-bcdc-07dd26f2a9a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.242480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddbf83c-b6f8-4485-bcdc-07dd26f2a9a3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-cddbf83c-b6f8-4485-bcdc-07dd26f2a9a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:25.267107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:25.267985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:25.268861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:25.268861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:25.268861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:25.268861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:25.268861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 257/518] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:25.269994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbf83c-b6f8-4485-bcdc-07dd26f2a9a3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.273739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:25.311206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:25.335331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b9487-fb4f-40ef-8bae-b73753738a35 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.371207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44b2d8-d34f-4792-b1bb-3b39fd5bebb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.372014 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44b2d8-d34f-4792-b1bb-3b39fd5bebb8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.372303 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44b2d8-d34f-4792-b1bb-3b39fd5bebb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.372590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f44b2d8-d34f-4792-b1bb-3b39fd5bebb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.384815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddbf83c-b6f8-4485-bcdc-07dd26f2a9a3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.386715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44b2d8-d34f-4792-b1bb-3b39fd5bebb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.388310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f44b2d8-d34f-4792-b1bb-3b39fd5bebb8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0f44b2d8-d34f-4792-b1bb-3b39fd5bebb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:25.408525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:25.409458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:25.410316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:25.410316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:25.410316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:25.410316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:25.410316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 262/519] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:25.411079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44b2d8-d34f-4792-b1bb-3b39fd5bebb8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.424618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88021002-2dc7-4e0d-bce3-662f8a5a6699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.425616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88021002-2dc7-4e0d-bce3-662f8a5a6699 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.425906 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88021002-2dc7-4e0d-bce3-662f8a5a6699 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.426340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88021002-2dc7-4e0d-bce3-662f8a5a6699 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.501951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88021002-2dc7-4e0d-bce3-662f8a5a6699 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.503322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88021002-2dc7-4e0d-bce3-662f8a5a6699 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.517352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:25.524971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f44b2d8-d34f-4792-b1bb-3b39fd5bebb8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.539696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:25.564971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d669ce5-e3d3-4229-b311-100ec693907b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.566115 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d669ce5-e3d3-4229-b311-100ec693907b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.566549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d669ce5-e3d3-4229-b311-100ec693907b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.566966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d669ce5-e3d3-4229-b311-100ec693907b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.577428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d669ce5-e3d3-4229-b311-100ec693907b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.578952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d669ce5-e3d3-4229-b311-100ec693907b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7d669ce5-e3d3-4229-b311-100ec693907b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:25.594502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:25.595376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:25.596211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:25.596211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:25.596211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:25.596211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:25.596211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:25.596211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:25.596211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:25.596211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:25.596211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 263/520] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:25.597320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d669ce5-e3d3-4229-b311-100ec693907b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.623564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 258/521] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:25.627091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88021002-2dc7-4e0d-bce3-662f8a5a6699 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.703768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d669ce5-e3d3-4229-b311-100ec693907b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.731336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88021002-2dc7-4e0d-bce3-662f8a5a6699 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.737049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c4400-9af6-4648-a757-4035d89ff8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.737751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c4400-9af6-4648-a757-4035d89ff8fa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.738040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c4400-9af6-4648-a757-4035d89ff8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.738320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c4400-9af6-4648-a757-4035d89ff8fa None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.739051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c4400-9af6-4648-a757-4035d89ff8fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:25.739364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c4400-9af6-4648-a757-4035d89ff8fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.740741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b98c4400-9af6-4648-a757-4035d89ff8fa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.757028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 264/522] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:25.757028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c4400-9af6-4648-a757-4035d89ff8fa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.771861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:25.775446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59852321-d9d9-4e0e-9efc-ab9756815e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.776121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852321-d9d9-4e0e-9efc-ab9756815e3d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.776400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852321-d9d9-4e0e-9efc-ab9756815e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.776681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59852321-d9d9-4e0e-9efc-ab9756815e3d None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.777524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59852321-d9d9-4e0e-9efc-ab9756815e3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:25.777743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59852321-d9d9-4e0e-9efc-ab9756815e3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.779125 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59852321-d9d9-4e0e-9efc-ab9756815e3d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.781476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:25.792601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 259/523] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:25.793429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59852321-d9d9-4e0e-9efc-ab9756815e3d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.866785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c4400-9af6-4648-a757-4035d89ff8fa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.904197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb850402-1843-40f2-bf61-2b69cd1e1738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.904929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb850402-1843-40f2-bf61-2b69cd1e1738 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.905210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb850402-1843-40f2-bf61-2b69cd1e1738 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.905528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb850402-1843-40f2-bf61-2b69cd1e1738 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.906884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb850402-1843-40f2-bf61-2b69cd1e1738 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:25.907197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb850402-1843-40f2-bf61-2b69cd1e1738 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.907288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59852321-d9d9-4e0e-9efc-ab9756815e3d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:25.908649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb850402-1843-40f2-bf61-2b69cd1e1738 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.927063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 265/524] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:25.927874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb850402-1843-40f2-bf61-2b69cd1e1738 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:25.946670 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf78b9-3199-4c3d-978f-472012c08887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:25.947687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf78b9-3199-4c3d-978f-472012c08887 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:25.948000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf78b9-3199-4c3d-978f-472012c08887 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:25.948276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cf78b9-3199-4c3d-978f-472012c08887 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:25.960233 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf78b9-3199-4c3d-978f-472012c08887 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:25.961708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cf78b9-3199-4c3d-978f-472012c08887 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-02cf78b9-3199-4c3d-978f-472012c08887 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:25.979385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:25.980379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:25.981195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:25.981195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:25.981195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:25.981195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:25.981195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 260/525] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:25.982015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf78b9-3199-4c3d-978f-472012c08887 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:26.013294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:26.029907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:26.039083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb850402-1843-40f2-bf61-2b69cd1e1738 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.074481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b93cd-cfbc-4d9a-b7ce-406ebbc2512b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.075273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b93cd-cfbc-4d9a-b7ce-406ebbc2512b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.075556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b93cd-cfbc-4d9a-b7ce-406ebbc2512b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.075829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b93cd-cfbc-4d9a-b7ce-406ebbc2512b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.083472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b93cd-cfbc-4d9a-b7ce-406ebbc2512b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.084930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b93cd-cfbc-4d9a-b7ce-406ebbc2512b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:26.092879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cf78b9-3199-4c3d-978f-472012c08887 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c29b93cd-cfbc-4d9a-b7ce-406ebbc2512b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:26.099889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:26.100691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 266/526] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:26.102351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b93cd-cfbc-4d9a-b7ce-406ebbc2512b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:26.135281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f58a9-4452-47ba-9033-59e15a9053b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.136083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f58a9-4452-47ba-9033-59e15a9053b1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.136371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f58a9-4452-47ba-9033-59e15a9053b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.136647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f58a9-4452-47ba-9033-59e15a9053b1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.145114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f58a9-4452-47ba-9033-59e15a9053b1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.146590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f58a9-4452-47ba-9033-59e15a9053b1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f31f58a9-4452-47ba-9033-59e15a9053b1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:26.161602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:26.162484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:26.163349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 261/527] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:25 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:26.163997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f58a9-4452-47ba-9033-59e15a9053b1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:26.216403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b93cd-cfbc-4d9a-b7ce-406ebbc2512b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.257288 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b36c6-c025-4ba5-899e-6e2185e1bb4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.257978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b36c6-c025-4ba5-899e-6e2185e1bb4c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.258559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b36c6-c025-4ba5-899e-6e2185e1bb4c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.258559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b36c6-c025-4ba5-899e-6e2185e1bb4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.259391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b36c6-c025-4ba5-899e-6e2185e1bb4c None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=1cc0050f14664630a36814ca670f124f) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:26.259709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b36c6-c025-4ba5-899e-6e2185e1bb4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.261157 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2b36c6-c025-4ba5-899e-6e2185e1bb4c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:26.264814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:26.282453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 267/528] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:26 2022] GET /identity/v3/role_assignments?user.id=1cc0050f14664630a36814ca670f124f&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:26.283848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b36c6-c025-4ba5-899e-6e2185e1bb4c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:26.301016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f58a9-4452-47ba-9033-59e15a9053b1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.309297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:26.343764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f709a-6072-4f9b-9873-e52b920bb82b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.344590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f709a-6072-4f9b-9873-e52b920bb82b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.344950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f709a-6072-4f9b-9873-e52b920bb82b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.345304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3f709a-6072-4f9b-9873-e52b920bb82b None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.346175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f709a-6072-4f9b-9873-e52b920bb82b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:26.346557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f709a-6072-4f9b-9873-e52b920bb82b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.348067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3f709a-6072-4f9b-9873-e52b920bb82b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:26.366829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 262/529] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:26.367980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f709a-6072-4f9b-9873-e52b920bb82b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:26.427400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b36c6-c025-4ba5-899e-6e2185e1bb4c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.476311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0c5ce-9dd2-4b59-9f90-6eb9b7f43826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.477369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0c5ce-9dd2-4b59-9f90-6eb9b7f43826 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.477972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0c5ce-9dd2-4b59-9f90-6eb9b7f43826 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.478417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb0c5ce-9dd2-4b59-9f90-6eb9b7f43826 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.479923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0c5ce-9dd2-4b59-9f90-6eb9b7f43826 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:26.480379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0c5ce-9dd2-4b59-9f90-6eb9b7f43826 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.482728 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb0c5ce-9dd2-4b59-9f90-6eb9b7f43826 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:26.485821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3f709a-6072-4f9b-9873-e52b920bb82b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.498891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 268/530] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:26.500147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0c5ce-9dd2-4b59-9f90-6eb9b7f43826 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:26.522711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1156998-5f45-44ad-853a-29431846778e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.523395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1156998-5f45-44ad-853a-29431846778e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.523782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1156998-5f45-44ad-853a-29431846778e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.524161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1156998-5f45-44ad-853a-29431846778e None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.525120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1156998-5f45-44ad-853a-29431846778e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:26.525577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1156998-5f45-44ad-853a-29431846778e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.527144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1156998-5f45-44ad-853a-29431846778e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:26.566816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:26.576137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 263/531] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Jul 12 13:14:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:26.577161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1156998-5f45-44ad-853a-29431846778e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:26.605860 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:26.616420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb0c5ce-9dd2-4b59-9f90-6eb9b7f43826 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.652761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf938518-01fa-4755-ba05-107ee3cbc52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.653580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf938518-01fa-4755-ba05-107ee3cbc52b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.653866 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf938518-01fa-4755-ba05-107ee3cbc52b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.654142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf938518-01fa-4755-ba05-107ee3cbc52b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.661263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf938518-01fa-4755-ba05-107ee3cbc52b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.662747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf938518-01fa-4755-ba05-107ee3cbc52b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-cf938518-01fa-4755-ba05-107ee3cbc52b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:26.677850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:26.678907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:26.679960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:26.679960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:26.679960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:26.679960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:26.679960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:26.679960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:26.679960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:26.679960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:26.683683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 269/532] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:26.684545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf938518-01fa-4755-ba05-107ee3cbc52b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:26.693115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1156998-5f45-44ad-853a-29431846778e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.732372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27520dfb-ad0f-4a4c-8c7e-11ff7683842f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.733198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27520dfb-ad0f-4a4c-8c7e-11ff7683842f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.733621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27520dfb-ad0f-4a4c-8c7e-11ff7683842f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.734533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27520dfb-ad0f-4a4c-8c7e-11ff7683842f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.736403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27520dfb-ad0f-4a4c-8c7e-11ff7683842f None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=0ff10db6937b4f2b889c30dd5fad8111) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:26.736833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27520dfb-ad0f-4a4c-8c7e-11ff7683842f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.738572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27520dfb-ad0f-4a4c-8c7e-11ff7683842f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:26.745431 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 264/533] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:26 2022] GET /identity/v3/role_assignments?user.id=0ff10db6937b4f2b889c30dd5fad8111&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:26.746278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27520dfb-ad0f-4a4c-8c7e-11ff7683842f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:26.797886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf938518-01fa-4755-ba05-107ee3cbc52b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.823357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:26.841158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf0843-437d-43dc-acb1-23a7fa2d142a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.841776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf0843-437d-43dc-acb1-23a7fa2d142a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.842053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf0843-437d-43dc-acb1-23a7fa2d142a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.842323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf0843-437d-43dc-acb1-23a7fa2d142a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.886792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:26.922130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27520dfb-ad0f-4a4c-8c7e-11ff7683842f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:26.968355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf0843-437d-43dc-acb1-23a7fa2d142a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.971851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf0843-437d-43dc-acb1-23a7fa2d142a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:26.991851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354fb09-1d0d-4874-9e8b-7e555b8c37db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:26.992880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354fb09-1d0d-4874-9e8b-7e555b8c37db None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:26.993416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354fb09-1d0d-4874-9e8b-7e555b8c37db None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:26.993900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c354fb09-1d0d-4874-9e8b-7e555b8c37db None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:26.995333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354fb09-1d0d-4874-9e8b-7e555b8c37db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:26.995904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354fb09-1d0d-4874-9e8b-7e555b8c37db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:26.997588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c354fb09-1d0d-4874-9e8b-7e555b8c37db None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:27.025697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 265/534] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:27.032146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354fb09-1d0d-4874-9e8b-7e555b8c37db None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:27.125261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 270/535] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:27.131588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf0843-437d-43dc-acb1-23a7fa2d142a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:27.166915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:27.240415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf0843-437d-43dc-acb1-23a7fa2d142a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:27.251667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c354fb09-1d0d-4874-9e8b-7e555b8c37db None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:27.276803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d4fc70-2824-49f9-95fd-e0ad88f33c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:27.277627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4fc70-2824-49f9-95fd-e0ad88f33c87 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:27.277916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4fc70-2824-49f9-95fd-e0ad88f33c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:27.278297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d4fc70-2824-49f9-95fd-e0ad88f33c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:27.289981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4fc70-2824-49f9-95fd-e0ad88f33c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:27.291447 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d4fc70-2824-49f9-95fd-e0ad88f33c87 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:27.295428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3af85-8007-48ce-8f7a-71948a39c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:27.296089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3af85-8007-48ce-8f7a-71948a39c82c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:27.296421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3af85-8007-48ce-8f7a-71948a39c82c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:27.297763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3af85-8007-48ce-8f7a-71948a39c82c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:27.301398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3af85-8007-48ce-8f7a-71948a39c82c None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b4f12d08d7084387a1bea7865bf77d0e) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:27.301398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3af85-8007-48ce-8f7a-71948a39c82c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:27.301398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3af85-8007-48ce-8f7a-71948a39c82c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:27.307211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 266/536] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:27 2022] GET /identity/v3/role_assignments?user.id=b4f12d08d7084387a1bea7865bf77d0e&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:27.308624 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643ee46-94ba-42ba-97f9-e8d5a329a4cb None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:27.308919 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643ee46-94ba-42ba-97f9-e8d5a329a4cb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:27.309201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643ee46-94ba-42ba-97f9-e8d5a329a4cb None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b6d4fc70-2824-49f9-95fd-e0ad88f33c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:27.310331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:27.311442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:27.313135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:27.313135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:27.313135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:27.313135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:27.313135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 271/538] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:27.317740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 267/538] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:27.319283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd26a6-359e-4c26-b71f-7bfedcb3b4ba None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:27.319580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd26a6-359e-4c26-b71f-7bfedcb3b4ba None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:27.319855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbd26a6-359e-4c26-b71f-7bfedcb3b4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:27.322156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643ee46-94ba-42ba-97f9-e8d5a329a4cb None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:27.342486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:27.343071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6bbd26a6-359e-4c26-b71f-7bfedcb3b4ba None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:27.345936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6bbd26a6-359e-4c26-b71f-7bfedcb3b4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:27.474764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 272/539] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:27.501659 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd26a6-359e-4c26-b71f-7bfedcb3b4ba None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:27.503954 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:27.542635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643ee46-94ba-42ba-97f9-e8d5a329a4cb None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:27.615457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cebd21-46a8-4f76-b468-28eeb28de762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:27.616135 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cebd21-46a8-4f76-b468-28eeb28de762 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:27.616417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cebd21-46a8-4f76-b468-28eeb28de762 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:27.616689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cebd21-46a8-4f76-b468-28eeb28de762 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:27.617448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cebd21-46a8-4f76-b468-28eeb28de762 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:27.617757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cebd21-46a8-4f76-b468-28eeb28de762 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:27.619214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cebd21-46a8-4f76-b468-28eeb28de762 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:27.642891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 268/540] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:27.654071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cebd21-46a8-4f76-b468-28eeb28de762 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:27.676057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbd26a6-359e-4c26-b71f-7bfedcb3b4ba None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:27.746023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea38f82-888e-4d45-abc5-43b0e9949523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:27.746740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea38f82-888e-4d45-abc5-43b0e9949523 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:27.747021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea38f82-888e-4d45-abc5-43b0e9949523 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:27.747294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea38f82-888e-4d45-abc5-43b0e9949523 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:27.763799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea38f82-888e-4d45-abc5-43b0e9949523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6ea38f82-888e-4d45-abc5-43b0e9949523 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:27.775433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:27.776334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:27.777121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:27.785028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 273/541] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:27.787977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea38f82-888e-4d45-abc5-43b0e9949523 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:27.793219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:27.843925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cebd21-46a8-4f76-b468-28eeb28de762 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:27.898415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65653-55c1-432f-a927-d9fec2acbe29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:27.899781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65653-55c1-432f-a927-d9fec2acbe29 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:27.900277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65653-55c1-432f-a927-d9fec2acbe29 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:27.901060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a65653-55c1-432f-a927-d9fec2acbe29 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:27.902935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:27.916322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a65653-55c1-432f-a927-d9fec2acbe29 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:27.919099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a65653-55c1-432f-a927-d9fec2acbe29 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-69a65653-55c1-432f-a927-d9fec2acbe29 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:27.950845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:27.951737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:27.952664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:27.952664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:27.952664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:27.952664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:27.952664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:27.952664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:27.952664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:27.952664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:27.958601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 269/542] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:27.964046 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65653-55c1-432f-a927-d9fec2acbe29 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:27.985272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea38f82-888e-4d45-abc5-43b0e9949523 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:28.047490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee894e-94e9-4056-bead-78662efd9319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:28.048496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee894e-94e9-4056-bead-78662efd9319 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:28.049044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee894e-94e9-4056-bead-78662efd9319 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:28.049537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ee894e-94e9-4056-bead-78662efd9319 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:28.050530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee894e-94e9-4056-bead-78662efd9319 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:28.052480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee894e-94e9-4056-bead-78662efd9319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:28.054154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ee894e-94e9-4056-bead-78662efd9319 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:28.075476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 274/543] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:28.081266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee894e-94e9-4056-bead-78662efd9319 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:28.138939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a65653-55c1-432f-a927-d9fec2acbe29 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:28.175693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866625d2-a9b0-47b1-acc7-4532096f3e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:28.176698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866625d2-a9b0-47b1-acc7-4532096f3e5f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:28.177203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866625d2-a9b0-47b1-acc7-4532096f3e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:28.177700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866625d2-a9b0-47b1-acc7-4532096f3e5f None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:28.178698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866625d2-a9b0-47b1-acc7-4532096f3e5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:28.179216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866625d2-a9b0-47b1-acc7-4532096f3e5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:28.181060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866625d2-a9b0-47b1-acc7-4532096f3e5f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:28.183162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:28.194418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 270/544] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:28.200832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866625d2-a9b0-47b1-acc7-4532096f3e5f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:28.322450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ee894e-94e9-4056-bead-78662efd9319 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:28.368297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:28.381887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e0f9d-e2c2-45e4-8f8f-82cb3f0fbcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:28.383809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e0f9d-e2c2-45e4-8f8f-82cb3f0fbcec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:28.385748 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e0f9d-e2c2-45e4-8f8f-82cb3f0fbcec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:28.386247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4e0f9d-e2c2-45e4-8f8f-82cb3f0fbcec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:28.410064 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e0f9d-e2c2-45e4-8f8f-82cb3f0fbcec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:28.412430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4e0f9d-e2c2-45e4-8f8f-82cb3f0fbcec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:28.416575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866625d2-a9b0-47b1-acc7-4532096f3e5f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2a4e0f9d-e2c2-45e4-8f8f-82cb3f0fbcec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:28.453397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:28.454274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:28.455087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:28.455087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:28.455087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:28.455087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:28.462559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 275/545] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:28.466597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e0f9d-e2c2-45e4-8f8f-82cb3f0fbcec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:28.490406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b1bb16-7021-46ab-955a-494544ce6b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:28.492908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1bb16-7021-46ab-955a-494544ce6b0e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:28.493457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1bb16-7021-46ab-955a-494544ce6b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:28.494109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b1bb16-7021-46ab-955a-494544ce6b0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:28.495824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1bb16-7021-46ab-955a-494544ce6b0e None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=83837e4f8f9b4e538658876cfb56db16) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:28.497623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1bb16-7021-46ab-955a-494544ce6b0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:28.501447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b1bb16-7021-46ab-955a-494544ce6b0e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:28.515756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 271/546] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:28 2022] GET /identity/v3/role_assignments?user.id=83837e4f8f9b4e538658876cfb56db16&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 259 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:28.604858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4e0f9d-e2c2-45e4-8f8f-82cb3f0fbcec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:28.628076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdd5be-1bde-43b6-93d9-ab28a3759e6f None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:28.628694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdd5be-1bde-43b6-93d9-ab28a3759e6f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:28.629167 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdd5be-1bde-43b6-93d9-ab28a3759e6f None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:28.630850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 272/547] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:14:28.649657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f1d0f-73d2-43a3-8198-2da968f43caa None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:28.649983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f1d0f-73d2-43a3-8198-2da968f43caa None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:28.650224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1f1d0f-73d2-43a3-8198-2da968f43caa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:28.650816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb35e0a-f60a-4131-bd80-8ca3a46fa57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:28.651691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb35e0a-f60a-4131-bd80-8ca3a46fa57b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:28.652179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb35e0a-f60a-4131-bd80-8ca3a46fa57b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:28.652648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb35e0a-f60a-4131-bd80-8ca3a46fa57b None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:28.653623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb35e0a-f60a-4131-bd80-8ca3a46fa57b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:28.654134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb35e0a-f60a-4131-bd80-8ca3a46fa57b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:28.655866 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb35e0a-f60a-4131-bd80-8ca3a46fa57b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:28.669992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 276/548] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:28.682806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb35e0a-f60a-4131-bd80-8ca3a46fa57b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:28.701866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5e1f1d0f-73d2-43a3-8198-2da968f43caa None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:28.706260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5e1f1d0f-73d2-43a3-8198-2da968f43caa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:28.751946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:28.791267 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:28.812752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 273/549] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:28.850420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f1d0f-73d2-43a3-8198-2da968f43caa None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:28.866726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb35e0a-f60a-4131-bd80-8ca3a46fa57b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:28.950657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1a1c9-72e8-4b75-a4f2-c10dce8b408a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:28.951478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1a1c9-72e8-4b75-a4f2-c10dce8b408a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:28.951769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1a1c9-72e8-4b75-a4f2-c10dce8b408a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:28.952050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d1a1c9-72e8-4b75-a4f2-c10dce8b408a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:28.970164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1a1c9-72e8-4b75-a4f2-c10dce8b408a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:28.971575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d1a1c9-72e8-4b75-a4f2-c10dce8b408a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:28.977326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1f1d0f-73d2-43a3-8198-2da968f43caa None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-06d1a1c9-72e8-4b75-a4f2-c10dce8b408a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:29.014402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:29.016277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:29.017323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 277/550] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:29.025766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1a1c9-72e8-4b75-a4f2-c10dce8b408a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:29.033060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0c41-e240-41eb-8430-f059767b255e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:29.034204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0c41-e240-41eb-8430-f059767b255e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.035486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0c41-e240-41eb-8430-f059767b255e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.035966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db0c41-e240-41eb-8430-f059767b255e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.059158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db0c41-e240-41eb-8430-f059767b255e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-26db0c41-e240-41eb-8430-f059767b255e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:29.069487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:29.070396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:29.071219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:29.074012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 274/551] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:29.080478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0c41-e240-41eb-8430-f059767b255e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:29.160106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:29.204232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:29.214449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d1a1c9-72e8-4b75-a4f2-c10dce8b408a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:29.234462 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db0c41-e240-41eb-8430-f059767b255e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:29.256273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd950fd5-8083-481e-a8e8-48593bd27f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:29.257158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950fd5-8083-481e-a8e8-48593bd27f0a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.257615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950fd5-8083-481e-a8e8-48593bd27f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.258033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd950fd5-8083-481e-a8e8-48593bd27f0a None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.259083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd950fd5-8083-481e-a8e8-48593bd27f0a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:29.259591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd950fd5-8083-481e-a8e8-48593bd27f0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:29.261232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd950fd5-8083-481e-a8e8-48593bd27f0a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:29.278786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0e7351-be29-4a8e-99d7-a2dd768ef65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:29.279482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e7351-be29-4a8e-99d7-a2dd768ef65b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.279739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e7351-be29-4a8e-99d7-a2dd768ef65b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.280087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0e7351-be29-4a8e-99d7-a2dd768ef65b None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.280302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 278/552] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 12 13:14:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:29.281489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e7351-be29-4a8e-99d7-a2dd768ef65b None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:29.282737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cb704-aaf7-464b-879d-7dda095fb6bd None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.283222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cb704-aaf7-464b-879d-7dda095fb6bd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.283689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cb704-aaf7-464b-879d-7dda095fb6bd None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.289173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e7351-be29-4a8e-99d7-a2dd768ef65b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:29.289173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0e7351-be29-4a8e-99d7-a2dd768ef65b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:29.289509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 279/553] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:29.290481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cb704-aaf7-464b-879d-7dda095fb6bd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:29.299082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 275/554] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:29.301043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d291483-1742-455f-928e-8cf9700c4388 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.301337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d291483-1742-455f-928e-8cf9700c4388 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.301646 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d291483-1742-455f-928e-8cf9700c4388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.325112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5d291483-1742-455f-928e-8cf9700c4388 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:29.326341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5d291483-1742-455f-928e-8cf9700c4388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:29.417470 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cb704-aaf7-464b-879d-7dda095fb6bd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:29.418119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 276/555] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:29.419460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d291483-1742-455f-928e-8cf9700c4388 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:29.454860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9e322f-930c-4fec-9d6f-020c20dc7d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:29.455516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e322f-930c-4fec-9d6f-020c20dc7d87 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.455800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e322f-930c-4fec-9d6f-020c20dc7d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.456076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9e322f-930c-4fec-9d6f-020c20dc7d87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.479628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:29.490722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:29.537653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e322f-930c-4fec-9d6f-020c20dc7d87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:29.539119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9e322f-930c-4fec-9d6f-020c20dc7d87 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:29.548755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d291483-1742-455f-928e-8cf9700c4388 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:29.587989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb99b90-a743-4c58-93e2-10ad5ee8851c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:29.588776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb99b90-a743-4c58-93e2-10ad5ee8851c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.589057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb99b90-a743-4c58-93e2-10ad5ee8851c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.589328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb99b90-a743-4c58-93e2-10ad5ee8851c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.600226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb99b90-a743-4c58-93e2-10ad5ee8851c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:29.601683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb99b90-a743-4c58-93e2-10ad5ee8851c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9bb99b90-a743-4c58-93e2-10ad5ee8851c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:29.619873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:29.620789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:29.622080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:29.622080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:29.622080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:29.622080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:29.622080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 277/556] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:29.622364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb99b90-a743-4c58-93e2-10ad5ee8851c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:29.651848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 280/557] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:29.651848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86e5bd-dee9-47ef-a8c6-1e3c871a28c9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.651848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86e5bd-dee9-47ef-a8c6-1e3c871a28c9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.651848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae86e5bd-dee9-47ef-a8c6-1e3c871a28c9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.653591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 281/558] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:29.659037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86e5bd-dee9-47ef-a8c6-1e3c871a28c9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:29.722777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:29.725176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:29.736922 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb99b90-a743-4c58-93e2-10ad5ee8851c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:29.771699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae86e5bd-dee9-47ef-a8c6-1e3c871a28c9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:29.777027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5a898-ff56-42f9-aa2f-4e197a659d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:29.777854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5a898-ff56-42f9-aa2f-4e197a659d63 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.778137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5a898-ff56-42f9-aa2f-4e197a659d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.778409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d5a898-ff56-42f9-aa2f-4e197a659d63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.789567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d5a898-ff56-42f9-aa2f-4e197a659d63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-15d5a898-ff56-42f9-aa2f-4e197a659d63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:29.798637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:29.800016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 278/559] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:29.800833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5a898-ff56-42f9-aa2f-4e197a659d63 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:29.816704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a192010-65e0-4082-a1de-a51fb398fd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:29.817786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a192010-65e0-4082-a1de-a51fb398fd62 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.818084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a192010-65e0-4082-a1de-a51fb398fd62 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.818522 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a192010-65e0-4082-a1de-a51fb398fd62 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.819274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a192010-65e0-4082-a1de-a51fb398fd62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:29.819591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a192010-65e0-4082-a1de-a51fb398fd62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:29.821006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a192010-65e0-4082-a1de-a51fb398fd62 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:29.833199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 282/560] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:29.834469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fdbee-d8f3-4534-bddc-0419f8157b04 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.834763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fdbee-d8f3-4534-bddc-0419f8157b04 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.835046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1fdbee-d8f3-4534-bddc-0419f8157b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.853933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6b1fdbee-d8f3-4534-bddc-0419f8157b04 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:29.855210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6b1fdbee-d8f3-4534-bddc-0419f8157b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:29.918181 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d5a898-ff56-42f9-aa2f-4e197a659d63 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:29.933421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 283/561] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:29.933421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fdbee-d8f3-4534-bddc-0419f8157b04 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:29.973545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb0348-555b-4baa-ad9d-73cf5a1d634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:29.974348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb0348-555b-4baa-ad9d-73cf5a1d634f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:29.974639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb0348-555b-4baa-ad9d-73cf5a1d634f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:29.974916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb0348-555b-4baa-ad9d-73cf5a1d634f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:29.978914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:29.987359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb0348-555b-4baa-ad9d-73cf5a1d634f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:29.988810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb0348-555b-4baa-ad9d-73cf5a1d634f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-42cb0348-555b-4baa-ad9d-73cf5a1d634f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:30.006238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:30.007134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:30.008010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:30.008010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:30.008010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:30.008010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:30.008010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 279/562] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:30.009594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb0348-555b-4baa-ad9d-73cf5a1d634f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.011208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:30.043570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1fdbee-d8f3-4534-bddc-0419f8157b04 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.080809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8d12d-9c40-4b07-be64-fe3d1030cf83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.081823 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8d12d-9c40-4b07-be64-fe3d1030cf83 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.082327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8d12d-9c40-4b07-be64-fe3d1030cf83 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.082663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8d12d-9c40-4b07-be64-fe3d1030cf83 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.083485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8d12d-9c40-4b07-be64-fe3d1030cf83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:30.083863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8d12d-9c40-4b07-be64-fe3d1030cf83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.085319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8d12d-9c40-4b07-be64-fe3d1030cf83 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.099213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 284/563] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:30.099825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8d12d-9c40-4b07-be64-fe3d1030cf83 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.113676 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb0348-555b-4baa-ad9d-73cf5a1d634f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.147760 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add98622-e161-4800-8677-0b1bcc63e6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.148547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add98622-e161-4800-8677-0b1bcc63e6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.148829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add98622-e161-4800-8677-0b1bcc63e6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.149105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add98622-e161-4800-8677-0b1bcc63e6e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.157655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add98622-e161-4800-8677-0b1bcc63e6e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.159105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add98622-e161-4800-8677-0b1bcc63e6e2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-add98622-e161-4800-8677-0b1bcc63e6e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:30.174157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:30.175096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:30.176022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:30.179924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 280/564] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:30.180530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add98622-e161-4800-8677-0b1bcc63e6e2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.206318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8d12d-9c40-4b07-be64-fe3d1030cf83 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.214112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:30.244888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d51a6c-14d4-4c68-b942-056b15e6d8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.246045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d51a6c-14d4-4c68-b942-056b15e6d8ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.246537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d51a6c-14d4-4c68-b942-056b15e6d8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.247039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d51a6c-14d4-4c68-b942-056b15e6d8ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.250145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:30.259399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d51a6c-14d4-4c68-b942-056b15e6d8ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c0d51a6c-14d4-4c68-b942-056b15e6d8ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:30.267462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:30.268372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:30.269178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:30.273584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 285/565] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:30.274404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d51a6c-14d4-4c68-b942-056b15e6d8ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.294916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add98622-e161-4800-8677-0b1bcc63e6e2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.332020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179d8e3-159a-4021-b592-b546f342bab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.332736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179d8e3-159a-4021-b592-b546f342bab1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.333027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179d8e3-159a-4021-b592-b546f342bab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.333499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179d8e3-159a-4021-b592-b546f342bab1 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.334145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179d8e3-159a-4021-b592-b546f342bab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:30.334376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179d8e3-159a-4021-b592-b546f342bab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.335748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7179d8e3-159a-4021-b592-b546f342bab1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.348586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 281/566] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:30.349238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179d8e3-159a-4021-b592-b546f342bab1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.384123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d51a6c-14d4-4c68-b942-056b15e6d8ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.419051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbce20-7a65-4e11-b2a0-598b43e63431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.419846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbce20-7a65-4e11-b2a0-598b43e63431 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.420135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbce20-7a65-4e11-b2a0-598b43e63431 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.420416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fbce20-7a65-4e11-b2a0-598b43e63431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.431664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbce20-7a65-4e11-b2a0-598b43e63431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.433116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fbce20-7a65-4e11-b2a0-598b43e63431 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a5fbce20-7a65-4e11-b2a0-598b43e63431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:30.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:30.451658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:30.457717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:30.457717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:30.457717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:30.457717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:30.457717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 286/567] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:30.457717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbce20-7a65-4e11-b2a0-598b43e63431 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.460212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179d8e3-159a-4021-b592-b546f342bab1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.474703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:30.487891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:30.501883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed23c2-11b2-4811-bb88-ed9bb0488da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.502486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed23c2-11b2-4811-bb88-ed9bb0488da8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.502773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed23c2-11b2-4811-bb88-ed9bb0488da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.503057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed23c2-11b2-4811-bb88-ed9bb0488da8 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.503904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed23c2-11b2-4811-bb88-ed9bb0488da8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:30.504215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed23c2-11b2-4811-bb88-ed9bb0488da8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.505662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed23c2-11b2-4811-bb88-ed9bb0488da8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.525368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 282/568] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:30.526245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed23c2-11b2-4811-bb88-ed9bb0488da8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.567160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fbce20-7a65-4e11-b2a0-598b43e63431 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.607849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fe56a-515c-4d4a-b808-52aa81c81239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.609001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fe56a-515c-4d4a-b808-52aa81c81239 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.609494 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fe56a-515c-4d4a-b808-52aa81c81239 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.609977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09fe56a-515c-4d4a-b808-52aa81c81239 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.610795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fe56a-515c-4d4a-b808-52aa81c81239 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:30.611104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fe56a-515c-4d4a-b808-52aa81c81239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.612465 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09fe56a-515c-4d4a-b808-52aa81c81239 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.624790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 287/569] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:30.624790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fe56a-515c-4d4a-b808-52aa81c81239 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.633645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed23c2-11b2-4811-bb88-ed9bb0488da8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.670261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f866c5-a565-4405-b009-82c01c08b268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.671060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f866c5-a565-4405-b009-82c01c08b268 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.671378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f866c5-a565-4405-b009-82c01c08b268 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.671731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f866c5-a565-4405-b009-82c01c08b268 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.678796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f866c5-a565-4405-b009-82c01c08b268 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.680232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f866c5-a565-4405-b009-82c01c08b268 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-22f866c5-a565-4405-b009-82c01c08b268 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:30.694618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:30.695666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:30.696574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:30.696574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:30.696574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:30.696574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:30.696574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:30.696574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:30.696574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:30.696574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:30.696574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 283/570] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:30.697069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f866c5-a565-4405-b009-82c01c08b268 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.708180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:30.731864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09fe56a-515c-4d4a-b808-52aa81c81239 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.740886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:30.774567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b076e84-00df-4326-bc44-98397f85b638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.775176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b076e84-00df-4326-bc44-98397f85b638 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.775462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b076e84-00df-4326-bc44-98397f85b638 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.775740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b076e84-00df-4326-bc44-98397f85b638 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.776493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b076e84-00df-4326-bc44-98397f85b638 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:30.776801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b076e84-00df-4326-bc44-98397f85b638 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.778206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b076e84-00df-4326-bc44-98397f85b638 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.791436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 288/571] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:30.792090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b076e84-00df-4326-bc44-98397f85b638 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.806880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f866c5-a565-4405-b009-82c01c08b268 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.841429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd96e-b3c8-4bd1-95a1-9abd8cff4076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.842237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd96e-b3c8-4bd1-95a1-9abd8cff4076 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.842526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd96e-b3c8-4bd1-95a1-9abd8cff4076 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.842802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cd96e-b3c8-4bd1-95a1-9abd8cff4076 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.853547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cd96e-b3c8-4bd1-95a1-9abd8cff4076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.855011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cd96e-b3c8-4bd1-95a1-9abd8cff4076 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-857cd96e-b3c8-4bd1-95a1-9abd8cff4076 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:30.871956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:30.872933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:30.873848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:30.873848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:30.873848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:30.873848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:30.873848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 284/572] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:30.874139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd96e-b3c8-4bd1-95a1-9abd8cff4076 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.899796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b076e84-00df-4326-bc44-98397f85b638 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:30.932612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726c8d5-1d75-45e2-a524-6197d3e8e724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:30.933424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726c8d5-1d75-45e2-a524-6197d3e8e724 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:30.933713 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726c8d5-1d75-45e2-a524-6197d3e8e724 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:30.933992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726c8d5-1d75-45e2-a524-6197d3e8e724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:30.943362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:30.945771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726c8d5-1d75-45e2-a524-6197d3e8e724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:30.947226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726c8d5-1d75-45e2-a524-6197d3e8e724 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7726c8d5-1d75-45e2-a524-6197d3e8e724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:30.965454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:30.966337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:30.967220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:30.967220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:30.967220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:30.967220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:30.967220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 289/573] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:30.967498 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726c8d5-1d75-45e2-a524-6197d3e8e724 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:30.972413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:30.981962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cd96e-b3c8-4bd1-95a1-9abd8cff4076 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.014406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1294af9-8476-47ea-af6a-70a88c951b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.015112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1294af9-8476-47ea-af6a-70a88c951b42 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.016536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1294af9-8476-47ea-af6a-70a88c951b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.016821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1294af9-8476-47ea-af6a-70a88c951b42 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.017647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1294af9-8476-47ea-af6a-70a88c951b42 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:31.017958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1294af9-8476-47ea-af6a-70a88c951b42 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.019401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1294af9-8476-47ea-af6a-70a88c951b42 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.031298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 285/574] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:31.032533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366c591-28da-4789-9edb-545cea7a950d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.032825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366c591-28da-4789-9edb-545cea7a950d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.033100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f366c591-28da-4789-9edb-545cea7a950d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.034371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 286/575] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:31.035061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366c591-28da-4789-9edb-545cea7a950d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.070719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726c8d5-1d75-45e2-a524-6197d3e8e724 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.103536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74e0cf-2a2d-4bb3-9058-dabc84582bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.104329 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74e0cf-2a2d-4bb3-9058-dabc84582bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.104617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74e0cf-2a2d-4bb3-9058-dabc84582bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.104895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a74e0cf-2a2d-4bb3-9058-dabc84582bf4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.112369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74e0cf-2a2d-4bb3-9058-dabc84582bf4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.113845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a74e0cf-2a2d-4bb3-9058-dabc84582bf4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4a74e0cf-2a2d-4bb3-9058-dabc84582bf4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:31.130068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:31.131034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 290/576] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:31.131983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74e0cf-2a2d-4bb3-9058-dabc84582bf4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.140844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f366c591-28da-4789-9edb-545cea7a950d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.174184 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ac2c7-a0b1-47de-8292-5d3a49af1f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.174841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ac2c7-a0b1-47de-8292-5d3a49af1f0b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.175124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ac2c7-a0b1-47de-8292-5d3a49af1f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.175403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78ac2c7-a0b1-47de-8292-5d3a49af1f0b None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.176127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ac2c7-a0b1-47de-8292-5d3a49af1f0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:31.176436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ac2c7-a0b1-47de-8292-5d3a49af1f0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.177824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78ac2c7-a0b1-47de-8292-5d3a49af1f0b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.182624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:31.191362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 287/577] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:31.192155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ac2c7-a0b1-47de-8292-5d3a49af1f0b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.206241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:31.238516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a74e0cf-2a2d-4bb3-9058-dabc84582bf4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.273781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f502a-9ef1-4481-bd2b-a99c7f331023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.274443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f502a-9ef1-4481-bd2b-a99c7f331023 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.274725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f502a-9ef1-4481-bd2b-a99c7f331023 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.275002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f502a-9ef1-4481-bd2b-a99c7f331023 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.275722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f502a-9ef1-4481-bd2b-a99c7f331023 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:31.276031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f502a-9ef1-4481-bd2b-a99c7f331023 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.277437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f502a-9ef1-4481-bd2b-a99c7f331023 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.288449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 291/578] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:31.289586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222d965-1a35-44aa-9cf1-64f6e3551e20 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.289877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222d965-1a35-44aa-9cf1-64f6e3551e20 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.290154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222d965-1a35-44aa-9cf1-64f6e3551e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.299047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78ac2c7-a0b1-47de-8292-5d3a49af1f0b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.306151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d222d965-1a35-44aa-9cf1-64f6e3551e20 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:31.307354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d222d965-1a35-44aa-9cf1-64f6e3551e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:31.332280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21347a-2682-438b-86a9-4682f2032155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.332930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21347a-2682-438b-86a9-4682f2032155 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.333211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21347a-2682-438b-86a9-4682f2032155 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.333519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd21347a-2682-438b-86a9-4682f2032155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.334348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21347a-2682-438b-86a9-4682f2032155 None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b69264fe173045e08c282171cd7370af) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:31.334658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21347a-2682-438b-86a9-4682f2032155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.336069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd21347a-2682-438b-86a9-4682f2032155 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.341442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 288/579] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/role_assignments?user.id=b69264fe173045e08c282171cd7370af&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:31.342064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21347a-2682-438b-86a9-4682f2032155 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.385049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 292/580] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:31.386204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222d965-1a35-44aa-9cf1-64f6e3551e20 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.449792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:31.453943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:31.480005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd21347a-2682-438b-86a9-4682f2032155 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.514171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d222d965-1a35-44aa-9cf1-64f6e3551e20 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.524968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596bf6a-5dc3-4413-8ecd-082435c21023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.525673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596bf6a-5dc3-4413-8ecd-082435c21023 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.525952 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596bf6a-5dc3-4413-8ecd-082435c21023 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.526239 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596bf6a-5dc3-4413-8ecd-082435c21023 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.527085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596bf6a-5dc3-4413-8ecd-082435c21023 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:31.527397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596bf6a-5dc3-4413-8ecd-082435c21023 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.528825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596bf6a-5dc3-4413-8ecd-082435c21023 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.548470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 289/581] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:31.549218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596bf6a-5dc3-4413-8ecd-082435c21023 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.557076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6c529-d292-4eb4-ab49-6084740014a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.558152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6c529-d292-4eb4-ab49-6084740014a1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.558625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6c529-d292-4eb4-ab49-6084740014a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.558899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6c529-d292-4eb4-ab49-6084740014a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.567974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6c529-d292-4eb4-ab49-6084740014a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.569466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c6c529-d292-4eb4-ab49-6084740014a1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f0c6c529-d292-4eb4-ab49-6084740014a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:31.587993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:31.589046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 293/582] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:31.590516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6c529-d292-4eb4-ab49-6084740014a1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.690469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596bf6a-5dc3-4413-8ecd-082435c21023 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.694430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6c529-d292-4eb4-ab49-6084740014a1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.696169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:31.732694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3ec00-4143-453d-9777-b7503bd61931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.733650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3ec00-4143-453d-9777-b7503bd61931 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.734195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3ec00-4143-453d-9777-b7503bd61931 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.734422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3ec00-4143-453d-9777-b7503bd61931 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.739394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210441b-f977-450f-9d78-09dd02f74c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.740485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210441b-f977-450f-9d78-09dd02f74c3a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.740925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210441b-f977-450f-9d78-09dd02f74c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.741210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210441b-f977-450f-9d78-09dd02f74c3a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.743797 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:31.746227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3ec00-4143-453d-9777-b7503bd61931 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:31.752343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210441b-f977-450f-9d78-09dd02f74c3a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.753862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0210441b-f977-450f-9d78-09dd02f74c3a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0fd3ec00-4143-453d-9777-b7503bd61931 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:31.754880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:31.755752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:31.756591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 294/583] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:31.757611 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3ec00-4143-453d-9777-b7503bd61931 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-0210441b-f977-450f-9d78-09dd02f74c3a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:31.778270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:31.779456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:14:31.781216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:31.785237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 290/584] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:31.786149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210441b-f977-450f-9d78-09dd02f74c3a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.885720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3ec00-4143-453d-9777-b7503bd61931 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.915063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210441b-f977-450f-9d78-09dd02f74c3a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:31.930222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57d972-f591-430b-90a2-14e6a94079c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.931697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d972-f591-430b-90a2-14e6a94079c4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.932103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d972-f591-430b-90a2-14e6a94079c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.932478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57d972-f591-430b-90a2-14e6a94079c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.947468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57d972-f591-430b-90a2-14e6a94079c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.950289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57d972-f591-430b-90a2-14e6a94079c4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.958845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7126b20-a579-4bac-8364-c4247e0b6fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:31.959515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7126b20-a579-4bac-8364-c4247e0b6fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:31.959799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7126b20-a579-4bac-8364-c4247e0b6fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:31.960081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7126b20-a579-4bac-8364-c4247e0b6fc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:31.960882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7126b20-a579-4bac-8364-c4247e0b6fc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:31.961194 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7126b20-a579-4bac-8364-c4247e0b6fc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:31.963092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7126b20-a579-4bac-8364-c4247e0b6fc1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2f57d972-f591-430b-90a2-14e6a94079c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:31.979158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:31.980126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:31.987318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:31.987318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:31.987318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:31.987318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:31.987577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 291/585] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:31.987651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 295/586] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:31.987651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57d972-f591-430b-90a2-14e6a94079c4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:31.987773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7126b20-a579-4bac-8364-c4247e0b6fc1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:32.002760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:32.045325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:32.092500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7126b20-a579-4bac-8364-c4247e0b6fc1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:32.098750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57d972-f591-430b-90a2-14e6a94079c4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:32.128393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abb124-854d-4010-8eca-1b706b6055e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:32.129058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abb124-854d-4010-8eca-1b706b6055e0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:32.129334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abb124-854d-4010-8eca-1b706b6055e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:32.129661 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13abb124-854d-4010-8eca-1b706b6055e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:32.130415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abb124-854d-4010-8eca-1b706b6055e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:32.130725 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abb124-854d-4010-8eca-1b706b6055e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:32.132079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13abb124-854d-4010-8eca-1b706b6055e0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:32.140818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 292/587] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:32.141498 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584526-c909-48ac-95cf-ca360f280220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:32.141645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abb124-854d-4010-8eca-1b706b6055e0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:32.142479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584526-c909-48ac-95cf-ca360f280220 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:32.142767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584526-c909-48ac-95cf-ca360f280220 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:32.143047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a584526-c909-48ac-95cf-ca360f280220 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:32.143896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a584526-c909-48ac-95cf-ca360f280220 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:32.144209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a584526-c909-48ac-95cf-ca360f280220 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:32.145833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a584526-c909-48ac-95cf-ca360f280220 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:32.169948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 296/588] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 12 13:14:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:32.169948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584526-c909-48ac-95cf-ca360f280220 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:32.240969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:32.258256 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13abb124-854d-4010-8eca-1b706b6055e0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:32.274067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a584526-c909-48ac-95cf-ca360f280220 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:32.275811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:32.297334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bed46-1a23-42c8-b1cd-910e5aa3b7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:32.298044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bed46-1a23-42c8-b1cd-910e5aa3b7ab None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:32.298320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bed46-1a23-42c8-b1cd-910e5aa3b7ab None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:32.298598 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bed46-1a23-42c8-b1cd-910e5aa3b7ab None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:32.299323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bed46-1a23-42c8-b1cd-910e5aa3b7ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:32.299630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bed46-1a23-42c8-b1cd-910e5aa3b7ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:32.301001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bed46-1a23-42c8-b1cd-910e5aa3b7ab None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:32.308550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6869b17-c3a5-4abf-b06f-bab2f3e5b608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:32.309467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6869b17-c3a5-4abf-b06f-bab2f3e5b608 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:32.309756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6869b17-c3a5-4abf-b06f-bab2f3e5b608 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:32.310033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6869b17-c3a5-4abf-b06f-bab2f3e5b608 None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/1cc0050f14664630a36814ca670f124f/roles/bd68bb224f0b4111bf407cf17e7143a4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:32.314746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 293/589] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:32.315285 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bed46-1a23-42c8-b1cd-910e5aa3b7ab None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:32.331494 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6869b17-c3a5-4abf-b06f-bab2f3e5b608 None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=1cc0050f14664630a36814ca670f124f, role_id=bd68bb224f0b4111bf407cf17e7143a4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:32.332963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6869b17-c3a5-4abf-b06f-bab2f3e5b608 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:32.351058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 297/590] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:14:32 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/1cc0050f14664630a36814ca670f124f/roles/bd68bb224f0b4111bf407cf17e7143a4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:32.352328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6869b17-c3a5-4abf-b06f-bab2f3e5b608 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:32.483512 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bed46-1a23-42c8-b1cd-910e5aa3b7ab None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:32.533402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6869b17-c3a5-4abf-b06f-bab2f3e5b608 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:32.545669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19992fab-02d2-471c-abd2-f1977ebf35a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:32.548405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19992fab-02d2-471c-abd2-f1977ebf35a3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:32.548405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19992fab-02d2-471c-abd2-f1977ebf35a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:32.548405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19992fab-02d2-471c-abd2-f1977ebf35a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:32.567350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19992fab-02d2-471c-abd2-f1977ebf35a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:32.569935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19992fab-02d2-471c-abd2-f1977ebf35a3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:32.578337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:32.596370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b4cdf-b3e5-4ee3-b85c-cdad3bc0524e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:32.597183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b4cdf-b3e5-4ee3-b85c-cdad3bc0524e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:32.597506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b4cdf-b3e5-4ee3-b85c-cdad3bc0524e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:32.598133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311b4cdf-b3e5-4ee3-b85c-cdad3bc0524e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-19992fab-02d2-471c-abd2-f1977ebf35a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:32.600806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:32.602442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:32.604173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:32.604173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:32.604173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:32.604173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:32.604173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 294/591] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:32.604173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19992fab-02d2-471c-abd2-f1977ebf35a3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:32.621201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b4cdf-b3e5-4ee3-b85c-cdad3bc0524e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:32.624935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311b4cdf-b3e5-4ee3-b85c-cdad3bc0524e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:32.638765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-311b4cdf-b3e5-4ee3-b85c-cdad3bc0524e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:32.670173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:32.671100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:32.672041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:32.672041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:32.672041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:32.672041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:32.673214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 298/592] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:32.675467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b4cdf-b3e5-4ee3-b85c-cdad3bc0524e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:32.743539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19992fab-02d2-471c-abd2-f1977ebf35a3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:32.786665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe72a99-db17-4e4a-b0f3-73c4cf94a889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:32.787909 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe72a99-db17-4e4a-b0f3-73c4cf94a889 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:32.788376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe72a99-db17-4e4a-b0f3-73c4cf94a889 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:32.788839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe72a99-db17-4e4a-b0f3-73c4cf94a889 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:32.798008 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe72a99-db17-4e4a-b0f3-73c4cf94a889 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:32.801519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe72a99-db17-4e4a-b0f3-73c4cf94a889 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9fe72a99-db17-4e4a-b0f3-73c4cf94a889 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:32.820861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:32.821814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:32.822704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:32.822704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:32.822704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:32.822704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:32.822704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:32.822704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:32.822704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:32.822704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:32.825548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 295/593] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:32.826860 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe72a99-db17-4e4a-b0f3-73c4cf94a889 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:32.843521 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311b4cdf-b3e5-4ee3-b85c-cdad3bc0524e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:32.890062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f5bc3-5f37-4612-a05b-adad960c7ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:32.891289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f5bc3-5f37-4612-a05b-adad960c7ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:32.891750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f5bc3-5f37-4612-a05b-adad960c7ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:32.892321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75f5bc3-5f37-4612-a05b-adad960c7ba3 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:32.894212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f5bc3-5f37-4612-a05b-adad960c7ba3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:32.894907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f5bc3-5f37-4612-a05b-adad960c7ba3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:32.898657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75f5bc3-5f37-4612-a05b-adad960c7ba3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:32.923174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 299/594] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:32.923174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f5bc3-5f37-4612-a05b-adad960c7ba3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:32.938838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:32.955872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe72a99-db17-4e4a-b0f3-73c4cf94a889 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:32.967123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:33.005115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e72d40-a5d8-42f0-b83e-7b6fbe9a45fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.006185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72d40-a5d8-42f0-b83e-7b6fbe9a45fb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.006669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72d40-a5d8-42f0-b83e-7b6fbe9a45fb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.007143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e72d40-a5d8-42f0-b83e-7b6fbe9a45fb None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.008081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e72d40-a5d8-42f0-b83e-7b6fbe9a45fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:33.008584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e72d40-a5d8-42f0-b83e-7b6fbe9a45fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:33.010184 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e72d40-a5d8-42f0-b83e-7b6fbe9a45fb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:33.023430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 296/595] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:33.024689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e72d40-a5d8-42f0-b83e-7b6fbe9a45fb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:33.041348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75f5bc3-5f37-4612-a05b-adad960c7ba3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.078869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c115b-fd0c-4929-a759-627e23b4245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.079540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c115b-fd0c-4929-a759-627e23b4245f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.079824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c115b-fd0c-4929-a759-627e23b4245f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.080100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c115b-fd0c-4929-a759-627e23b4245f None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.081014 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c115b-fd0c-4929-a759-627e23b4245f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:33.081332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c115b-fd0c-4929-a759-627e23b4245f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:33.082817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c115b-fd0c-4929-a759-627e23b4245f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:33.097408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 300/596] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:33.098388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c115b-fd0c-4929-a759-627e23b4245f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:33.130897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e72d40-a5d8-42f0-b83e-7b6fbe9a45fb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.166288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee1160-ca4b-46c6-a2d9-2ffa9eb1236c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.167461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee1160-ca4b-46c6-a2d9-2ffa9eb1236c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.167945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee1160-ca4b-46c6-a2d9-2ffa9eb1236c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.168427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee1160-ca4b-46c6-a2d9-2ffa9eb1236c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.176667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee1160-ca4b-46c6-a2d9-2ffa9eb1236c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:33.178455 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee1160-ca4b-46c6-a2d9-2ffa9eb1236c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-caee1160-ca4b-46c6-a2d9-2ffa9eb1236c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:33.194469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:33.195438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:14:33.196369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:33.202161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:33.203913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 297/597] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:33 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:33.204817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee1160-ca4b-46c6-a2d9-2ffa9eb1236c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:33.211379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c115b-fd0c-4929-a759-627e23b4245f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.216102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:33.259323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2d999-ae39-488d-aa61-f6a5b403d840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.260337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2d999-ae39-488d-aa61-f6a5b403d840 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.260789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2d999-ae39-488d-aa61-f6a5b403d840 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.261227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2d999-ae39-488d-aa61-f6a5b403d840 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.269360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2d999-ae39-488d-aa61-f6a5b403d840 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:33.271041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2d999-ae39-488d-aa61-f6a5b403d840 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-aee2d999-ae39-488d-aa61-f6a5b403d840 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:33.285891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:33.287061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:33.287956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:33.287956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:33.287956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:33.287956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:33.287956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:33.287956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:33.287956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:33.287956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:33.290286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 301/598] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:33.291246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2d999-ae39-488d-aa61-f6a5b403d840 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:33.316914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee1160-ca4b-46c6-a2d9-2ffa9eb1236c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.353113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e83c0-6a0e-4aa6-89c6-a972000e2a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.354145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e83c0-6a0e-4aa6-89c6-a972000e2a66 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.354492 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e83c0-6a0e-4aa6-89c6-a972000e2a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.354835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e83c0-6a0e-4aa6-89c6-a972000e2a66 None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/0ff10db6937b4f2b889c30dd5fad8111/roles/bd68bb224f0b4111bf407cf17e7143a4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.370560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e83c0-6a0e-4aa6-89c6-a972000e2a66 None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=0ff10db6937b4f2b889c30dd5fad8111, role_id=bd68bb224f0b4111bf407cf17e7143a4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:33.372126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e83c0-6a0e-4aa6-89c6-a972000e2a66 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:33.387593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 298/599] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:14:33 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/0ff10db6937b4f2b889c30dd5fad8111/roles/bd68bb224f0b4111bf407cf17e7143a4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:33.388789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e83c0-6a0e-4aa6-89c6-a972000e2a66 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:33.402722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2d999-ae39-488d-aa61-f6a5b403d840 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.445143 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312bd18-f619-4bd2-b5a2-94458c5106fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.446219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312bd18-f619-4bd2-b5a2-94458c5106fa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.446773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312bd18-f619-4bd2-b5a2-94458c5106fa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.447343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312bd18-f619-4bd2-b5a2-94458c5106fa None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.448482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312bd18-f619-4bd2-b5a2-94458c5106fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:33.449016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312bd18-f619-4bd2-b5a2-94458c5106fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:33.450898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6312bd18-f619-4bd2-b5a2-94458c5106fa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:33.472968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:33.473177 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 302/600] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Jul 12 13:14:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:33.473951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe73a1e-1daf-48c5-9adc-034933e6e836 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.474201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe73a1e-1daf-48c5-9adc-034933e6e836 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.474572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe73a1e-1daf-48c5-9adc-034933e6e836 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.478985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 303/601] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:33.482680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d15d06-7192-420b-9e0b-14c1ebb5f317 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.483091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d15d06-7192-420b-9e0b-14c1ebb5f317 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.483485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d15d06-7192-420b-9e0b-14c1ebb5f317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.504425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c7d15d06-7192-420b-9e0b-14c1ebb5f317 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:33.505994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c7d15d06-7192-420b-9e0b-14c1ebb5f317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:33.517357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:33.547306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e83c0-6a0e-4aa6-89c6-a972000e2a66 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.594633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 304/602] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:33.594633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d15d06-7192-420b-9e0b-14c1ebb5f317 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:33.600381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca434972-4b44-400a-99f9-3ad58471a84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.601202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca434972-4b44-400a-99f9-3ad58471a84c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.601666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca434972-4b44-400a-99f9-3ad58471a84c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.602044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca434972-4b44-400a-99f9-3ad58471a84c None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.602961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca434972-4b44-400a-99f9-3ad58471a84c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:33.603366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca434972-4b44-400a-99f9-3ad58471a84c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:33.604893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca434972-4b44-400a-99f9-3ad58471a84c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:33.620848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 299/603] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:33.622123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca434972-4b44-400a-99f9-3ad58471a84c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:33.720477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d15d06-7192-420b-9e0b-14c1ebb5f317 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.744069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca434972-4b44-400a-99f9-3ad58471a84c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.763344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:33.773620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbd5aa-1ef6-48a5-85d4-90953795f61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.774488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbd5aa-1ef6-48a5-85d4-90953795f61a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.775126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbd5aa-1ef6-48a5-85d4-90953795f61a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.775510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efbd5aa-1ef6-48a5-85d4-90953795f61a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.784454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836be670-1a19-44eb-ad00-c61d65721a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.785468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836be670-1a19-44eb-ad00-c61d65721a87 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.786111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836be670-1a19-44eb-ad00-c61d65721a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.786567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836be670-1a19-44eb-ad00-c61d65721a87 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.789794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbd5aa-1ef6-48a5-85d4-90953795f61a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:33.793274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efbd5aa-1ef6-48a5-85d4-90953795f61a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:33.796280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:33.802875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836be670-1a19-44eb-ad00-c61d65721a87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-836be670-1a19-44eb-ad00-c61d65721a87 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:33.810832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:33.812380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:33.813609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:33.815425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 300/604] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:33.816397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836be670-1a19-44eb-ad00-c61d65721a87 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4efbd5aa-1ef6-48a5-85d4-90953795f61a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:33.822500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:33.823567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:33.824692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:33.824692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:33.824692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:33.824692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:33.826888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 305/605] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:33.828062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbd5aa-1ef6-48a5-85d4-90953795f61a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:33.934700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836be670-1a19-44eb-ad00-c61d65721a87 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.965626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efbd5aa-1ef6-48a5-85d4-90953795f61a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:33.971756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a7f81-69b9-48a3-8685-d3250faa62a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:33.972830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7f81-69b9-48a3-8685-d3250faa62a9 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:33.973286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7f81-69b9-48a3-8685-d3250faa62a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:33.973727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85a7f81-69b9-48a3-8685-d3250faa62a9 None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b4f12d08d7084387a1bea7865bf77d0e/roles/bd68bb224f0b4111bf407cf17e7143a4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:33.993270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a7f81-69b9-48a3-8685-d3250faa62a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=b4f12d08d7084387a1bea7865bf77d0e, role_id=bd68bb224f0b4111bf407cf17e7143a4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:33.995615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85a7f81-69b9-48a3-8685-d3250faa62a9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:34.007979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ffec8-e211-4ed9-a191-6a908df19e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:34.008991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ffec8-e211-4ed9-a191-6a908df19e93 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:34.009279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ffec8-e211-4ed9-a191-6a908df19e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:34.009627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217ffec8-e211-4ed9-a191-6a908df19e93 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:34.010944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ffec8-e211-4ed9-a191-6a908df19e93 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:34.011302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ffec8-e211-4ed9-a191-6a908df19e93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:34.013091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217ffec8-e211-4ed9-a191-6a908df19e93 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:34.015927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 301/606] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:14:33 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b4f12d08d7084387a1bea7865bf77d0e/roles/bd68bb224f0b4111bf407cf17e7143a4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:34.017007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a7f81-69b9-48a3-8685-d3250faa62a9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:34.038269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 306/607] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:34.044565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ffec8-e211-4ed9-a191-6a908df19e93 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:34.086991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:34.114702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:34.166441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217ffec8-e211-4ed9-a191-6a908df19e93 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:34.204614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c79fcd-4afc-4a9c-8686-de232efd9f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:34.205716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c79fcd-4afc-4a9c-8686-de232efd9f40 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:34.206195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c79fcd-4afc-4a9c-8686-de232efd9f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:34.206613 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c79fcd-4afc-4a9c-8686-de232efd9f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:34.219413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c79fcd-4afc-4a9c-8686-de232efd9f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:34.221239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c79fcd-4afc-4a9c-8686-de232efd9f40 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-a1c79fcd-4afc-4a9c-8686-de232efd9f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:34.248151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:34.249159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:34.251144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:34.251144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:34.251144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:34.251144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:34.251144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 307/608] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:34.255645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c79fcd-4afc-4a9c-8686-de232efd9f40 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:34.267762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85a7f81-69b9-48a3-8685-d3250faa62a9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:34.317620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1cb14-7346-4eb2-81eb-e4e29dfb8703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:34.318219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1cb14-7346-4eb2-81eb-e4e29dfb8703 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:34.318509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1cb14-7346-4eb2-81eb-e4e29dfb8703 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:34.318787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1cb14-7346-4eb2-81eb-e4e29dfb8703 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:34.319529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1cb14-7346-4eb2-81eb-e4e29dfb8703 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:34.319842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1cb14-7346-4eb2-81eb-e4e29dfb8703 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:34.321219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1cb14-7346-4eb2-81eb-e4e29dfb8703 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:34.347196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 302/609] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:34.352410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1cb14-7346-4eb2-81eb-e4e29dfb8703 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:34.447219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c79fcd-4afc-4a9c-8686-de232efd9f40 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:34.468080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:34.509741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:34.521648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fbac5-59a7-4933-9e97-02f685b0382a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:34.522324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fbac5-59a7-4933-9e97-02f685b0382a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:34.522613 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fbac5-59a7-4933-9e97-02f685b0382a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:34.522894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989fbac5-59a7-4933-9e97-02f685b0382a None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:34.523632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fbac5-59a7-4933-9e97-02f685b0382a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:34.523943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fbac5-59a7-4933-9e97-02f685b0382a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:34.525319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989fbac5-59a7-4933-9e97-02f685b0382a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:34.539629 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1cb14-7346-4eb2-81eb-e4e29dfb8703 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:34.541947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 308/610] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:34.550385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fbac5-59a7-4933-9e97-02f685b0382a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:34.607506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe12089-58fc-4e09-828a-37b8a0af2dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:34.608236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe12089-58fc-4e09-828a-37b8a0af2dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:34.608524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe12089-58fc-4e09-828a-37b8a0af2dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:34.609394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe12089-58fc-4e09-828a-37b8a0af2dd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:34.627010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe12089-58fc-4e09-828a-37b8a0af2dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:34.628485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe12089-58fc-4e09-828a-37b8a0af2dd2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-6fe12089-58fc-4e09-828a-37b8a0af2dd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:34.657345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:34.658313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:34.659196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:34.659196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:34.659196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:34.659196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:34.659196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:34.659196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:34.659196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:34.659196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:34.660628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 303/611] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:34.665967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe12089-58fc-4e09-828a-37b8a0af2dd2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:34.728638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989fbac5-59a7-4933-9e97-02f685b0382a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:34.764726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755d89e4-8b31-4c5c-8278-30db96191fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:34.766137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d89e4-8b31-4c5c-8278-30db96191fcb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:34.766422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d89e4-8b31-4c5c-8278-30db96191fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:34.766706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755d89e4-8b31-4c5c-8278-30db96191fcb None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:34.774164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d89e4-8b31-4c5c-8278-30db96191fcb None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:34.775621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755d89e4-8b31-4c5c-8278-30db96191fcb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-755d89e4-8b31-4c5c-8278-30db96191fcb None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:34.791487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:34.792440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:34.793680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 309/612] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:34.794454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397f711-36f9-47bf-9ab3-3a43548d2838 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:34.794454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397f711-36f9-47bf-9ab3-3a43548d2838 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:34.794454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7397f711-36f9-47bf-9ab3-3a43548d2838 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:34.799902 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 310/613] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:34.801223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d1152-734e-4874-b8e0-44b1bcfbeac9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:34.801564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d1152-734e-4874-b8e0-44b1bcfbeac9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:34.801843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1d1152-734e-4874-b8e0-44b1bcfbeac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:34.810552 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe12089-58fc-4e09-828a-37b8a0af2dd2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:34.821657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ac1d1152-734e-4874-b8e0-44b1bcfbeac9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:34.823092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ac1d1152-734e-4874-b8e0-44b1bcfbeac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:34.849543 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:34.856944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a09a25-a03c-422e-938c-84936be3cef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:34.858083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a09a25-a03c-422e-938c-84936be3cef3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:34.858318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a09a25-a03c-422e-938c-84936be3cef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:34.858770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a09a25-a03c-422e-938c-84936be3cef3 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:34.860189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a09a25-a03c-422e-938c-84936be3cef3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:34.860504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a09a25-a03c-422e-938c-84936be3cef3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:34.861956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a09a25-a03c-422e-938c-84936be3cef3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:34.879169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 304/614] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:34.880357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a09a25-a03c-422e-938c-84936be3cef3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:34.896965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:34.910976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 311/615] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:34.912292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d1152-734e-4874-b8e0-44b1bcfbeac9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:35.009668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a09a25-a03c-422e-938c-84936be3cef3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:35.024638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1d1152-734e-4874-b8e0-44b1bcfbeac9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:35.051469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1d9f7-6254-4fb4-9abe-8018e43a2267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:35.052457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1d9f7-6254-4fb4-9abe-8018e43a2267 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.052743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1d9f7-6254-4fb4-9abe-8018e43a2267 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.053024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c1d9f7-6254-4fb4-9abe-8018e43a2267 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.054164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1d9f7-6254-4fb4-9abe-8018e43a2267 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:35.054475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1d9f7-6254-4fb4-9abe-8018e43a2267 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:35.056364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c1d9f7-6254-4fb4-9abe-8018e43a2267 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:35.065757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0b5da-b7f2-4fc2-a860-da76ea5d2f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:35.066992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0b5da-b7f2-4fc2-a860-da76ea5d2f57 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.067559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0b5da-b7f2-4fc2-a860-da76ea5d2f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.068141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f0b5da-b7f2-4fc2-a860-da76ea5d2f57 None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/83837e4f8f9b4e538658876cfb56db16/roles/bd68bb224f0b4111bf407cf17e7143a4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.078401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 305/616] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 12 13:14:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:35.079610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1d9f7-6254-4fb4-9abe-8018e43a2267 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:35.091483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0b5da-b7f2-4fc2-a860-da76ea5d2f57 None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=83837e4f8f9b4e538658876cfb56db16, role_id=bd68bb224f0b4111bf407cf17e7143a4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:35.093548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f0b5da-b7f2-4fc2-a860-da76ea5d2f57 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:35.117483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 312/617] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:14:34 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/83837e4f8f9b4e538658876cfb56db16/roles/bd68bb224f0b4111bf407cf17e7143a4 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:35.117483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0b5da-b7f2-4fc2-a860-da76ea5d2f57 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:35.128756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:35.203009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:35.238179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f0b5da-b7f2-4fc2-a860-da76ea5d2f57 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:35.281463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbba0f8-9dff-4551-9926-b656dcc62ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:35.282533 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbba0f8-9dff-4551-9926-b656dcc62ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.283019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbba0f8-9dff-4551-9926-b656dcc62ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.283855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbba0f8-9dff-4551-9926-b656dcc62ce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.288400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c1d9f7-6254-4fb4-9abe-8018e43a2267 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:35.297664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbba0f8-9dff-4551-9926-b656dcc62ce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:35.299380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbba0f8-9dff-4551-9926-b656dcc62ce6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0bbba0f8-9dff-4551-9926-b656dcc62ce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:35.322092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:35.323107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:35.323990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:35.323990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:35.323990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:35.323990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:35.326525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 313/618] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:35.333260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbba0f8-9dff-4551-9926-b656dcc62ce6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:35.390857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f29d41-61c0-4f5d-b875-71495cba4359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:35.391637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f29d41-61c0-4f5d-b875-71495cba4359 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.391885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f29d41-61c0-4f5d-b875-71495cba4359 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.392169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f29d41-61c0-4f5d-b875-71495cba4359 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.406960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:35.429522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f29d41-61c0-4f5d-b875-71495cba4359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-64f29d41-61c0-4f5d-b875-71495cba4359 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:35.439114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:35.440138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:35.441097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 306/619] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:35.448280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f29d41-61c0-4f5d-b875-71495cba4359 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:35.455126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbba0f8-9dff-4551-9926-b656dcc62ce6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:35.496573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c3ef-a574-47b9-9ab7-1f4ccd4201bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:35.497833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c3ef-a574-47b9-9ab7-1f4ccd4201bc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.498296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c3ef-a574-47b9-9ab7-1f4ccd4201bc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.498865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1690c3ef-a574-47b9-9ab7-1f4ccd4201bc None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.500361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690c3ef-a574-47b9-9ab7-1f4ccd4201bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:35.500846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690c3ef-a574-47b9-9ab7-1f4ccd4201bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:35.502816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1690c3ef-a574-47b9-9ab7-1f4ccd4201bc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:35.518046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 314/620] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:35.525052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c3ef-a574-47b9-9ab7-1f4ccd4201bc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:35.601539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f29d41-61c0-4f5d-b875-71495cba4359 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:35.631442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:35.647644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0142d42-b8d7-4842-9dea-c6af5d544a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:35.648255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0142d42-b8d7-4842-9dea-c6af5d544a1d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.648554 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0142d42-b8d7-4842-9dea-c6af5d544a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.648838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0142d42-b8d7-4842-9dea-c6af5d544a1d None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.649751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0142d42-b8d7-4842-9dea-c6af5d544a1d None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:35.650070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0142d42-b8d7-4842-9dea-c6af5d544a1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:35.651540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0142d42-b8d7-4842-9dea-c6af5d544a1d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:35.665811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 307/621] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:35.667109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ee34a-649a-4ada-a794-2d02a1b01c7b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.667410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ee34a-649a-4ada-a794-2d02a1b01c7b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.667691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ee34a-649a-4ada-a794-2d02a1b01c7b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.669083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 308/622] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:35.669819 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ee34a-649a-4ada-a794-2d02a1b01c7b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:35.689449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1690c3ef-a574-47b9-9ab7-1f4ccd4201bc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:35.716637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:35.735136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20907d74-9751-4e33-9edc-eb1fb1a529a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:35.737233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20907d74-9751-4e33-9edc-eb1fb1a529a1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.737894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20907d74-9751-4e33-9edc-eb1fb1a529a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.738320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20907d74-9751-4e33-9edc-eb1fb1a529a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.748014 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20907d74-9751-4e33-9edc-eb1fb1a529a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:35.749940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20907d74-9751-4e33-9edc-eb1fb1a529a1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-20907d74-9751-4e33-9edc-eb1fb1a529a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:35.769690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:35.770650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:35.771523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:35.771523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:35.771523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:35.771523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:35.771523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:35.771523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:35.771523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:35.771523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:35.778484 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 315/623] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:35.779410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db218f3-bc56-486c-a173-21863f6023c4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.779410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db218f3-bc56-486c-a173-21863f6023c4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.779410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db218f3-bc56-486c-a173-21863f6023c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.795612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ee34a-649a-4ada-a794-2d02a1b01c7b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:35.801807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9db218f3-bc56-486c-a173-21863f6023c4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:35.803305 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9db218f3-bc56-486c-a173-21863f6023c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:35.835170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f819-4118-4237-a7d7-5e4547a0fdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:35.836072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f819-4118-4237-a7d7-5e4547a0fdbb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:35.836363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f819-4118-4237-a7d7-5e4547a0fdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:35.836647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca4f819-4118-4237-a7d7-5e4547a0fdbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:35.849020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4f819-4118-4237-a7d7-5e4547a0fdbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:35.851015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca4f819-4118-4237-a7d7-5e4547a0fdbb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9ca4f819-4118-4237-a7d7-5e4547a0fdbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:35.870140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:35.871072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:35.871928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:35.871928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:35.871928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:35.871928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:35.871928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 309/624] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:35.872205 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f819-4118-4237-a7d7-5e4547a0fdbb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:35.891703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 316/625] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:35.892778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db218f3-bc56-486c-a173-21863f6023c4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:35.945983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:36.016419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca4f819-4118-4237-a7d7-5e4547a0fdbb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.031847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db218f3-bc56-486c-a173-21863f6023c4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.046486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:36.063250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf2eae-b03a-4acf-9d76-d0f18e53110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.064394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf2eae-b03a-4acf-9d76-d0f18e53110c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.064908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf2eae-b03a-4acf-9d76-d0f18e53110c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.065409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecf2eae-b03a-4acf-9d76-d0f18e53110c None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.066416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf2eae-b03a-4acf-9d76-d0f18e53110c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:36.067084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf2eae-b03a-4acf-9d76-d0f18e53110c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:36.068773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecf2eae-b03a-4acf-9d76-d0f18e53110c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:36.081629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f2b3f-81ff-44fa-a1df-1ddf5035be34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.082298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2b3f-81ff-44fa-a1df-1ddf5035be34 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.082583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2b3f-81ff-44fa-a1df-1ddf5035be34 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.082859 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f2b3f-81ff-44fa-a1df-1ddf5035be34 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.083584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f2b3f-81ff-44fa-a1df-1ddf5035be34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:36.083889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f2b3f-81ff-44fa-a1df-1ddf5035be34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:36.085399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355f2b3f-81ff-44fa-a1df-1ddf5035be34 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:36.085610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 310/626] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:36.086681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf2eae-b03a-4acf-9d76-d0f18e53110c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:36.106949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 317/627] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:36.106949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f2b3f-81ff-44fa-a1df-1ddf5035be34 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:36.200945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:36.204467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355f2b3f-81ff-44fa-a1df-1ddf5035be34 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.205471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecf2eae-b03a-4acf-9d76-d0f18e53110c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.239528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc6cfa-5460-4455-a8c9-57d68236ec8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.240481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc6cfa-5460-4455-a8c9-57d68236ec8c None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.240919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc6cfa-5460-4455-a8c9-57d68236ec8c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.241028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc6cfa-5460-4455-a8c9-57d68236ec8c None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b69264fe173045e08c282171cd7370af/roles/bd68bb224f0b4111bf407cf17e7143a4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.246270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f1dac-b6e6-4e11-8d4d-a192f2161502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.247401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f1dac-b6e6-4e11-8d4d-a192f2161502 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.247905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f1dac-b6e6-4e11-8d4d-a192f2161502 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.248410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3f1dac-b6e6-4e11-8d4d-a192f2161502 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.256914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc6cfa-5460-4455-a8c9-57d68236ec8c None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=b69264fe173045e08c282171cd7370af, role_id=bd68bb224f0b4111bf407cf17e7143a4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:36.258398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc6cfa-5460-4455-a8c9-57d68236ec8c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:36.261015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3f1dac-b6e6-4e11-8d4d-a192f2161502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9a3f1dac-b6e6-4e11-8d4d-a192f2161502 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:36.269633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:36.270536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:36.271433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:36.278387 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 311/629] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:36.279071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 318/629] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:14:36 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b69264fe173045e08c282171cd7370af/roles/bd68bb224f0b4111bf407cf17e7143a4 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:36.279207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f1dac-b6e6-4e11-8d4d-a192f2161502 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:36.282360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc6cfa-5460-4455-a8c9-57d68236ec8c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:36.321862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:36.408888 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3f1dac-b6e6-4e11-8d4d-a192f2161502 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.410388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc6cfa-5460-4455-a8c9-57d68236ec8c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.448337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9edfdec-1861-49f6-9fbb-fc4582b35934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.449032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edfdec-1861-49f6-9fbb-fc4582b35934 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.449437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edfdec-1861-49f6-9fbb-fc4582b35934 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.449724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9edfdec-1861-49f6-9fbb-fc4582b35934 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.450585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edfdec-1861-49f6-9fbb-fc4582b35934 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:36.450894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edfdec-1861-49f6-9fbb-fc4582b35934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:36.452988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9edfdec-1861-49f6-9fbb-fc4582b35934 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:36.454259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae765a1-fec7-4f41-abee-2f4d89d9085c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.455462 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae765a1-fec7-4f41-abee-2f4d89d9085c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.456045 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae765a1-fec7-4f41-abee-2f4d89d9085c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.456545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae765a1-fec7-4f41-abee-2f4d89d9085c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.463119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 319/630] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:36.467994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123978-1ee7-4172-be6d-0f75db8e2f43 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.467994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123978-1ee7-4172-be6d-0f75db8e2f43 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.467994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6123978-1ee7-4172-be6d-0f75db8e2f43 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.467994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 320/631] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:36.470040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123978-1ee7-4172-be6d-0f75db8e2f43 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:36.472037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae765a1-fec7-4f41-abee-2f4d89d9085c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:36.473582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae765a1-fec7-4f41-abee-2f4d89d9085c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-4ae765a1-fec7-4f41-abee-2f4d89d9085c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:36.490468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:36.491466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:36.492406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 312/632] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 12 13:14:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:36.493549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90738244-c9ec-43d1-8e2a-47c98604fc86 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.493836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90738244-c9ec-43d1-8e2a-47c98604fc86 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.494173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90738244-c9ec-43d1-8e2a-47c98604fc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.513804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-90738244-c9ec-43d1-8e2a-47c98604fc86 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:36.515904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-90738244-c9ec-43d1-8e2a-47c98604fc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:36.535838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:36.586427 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:36.613962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 313/633] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:36.615011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6123978-1ee7-4172-be6d-0f75db8e2f43 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.615368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90738244-c9ec-43d1-8e2a-47c98604fc86 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:36.661682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cb935-df62-4a19-97c2-2780a67cf200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.662583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cb935-df62-4a19-97c2-2780a67cf200 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.662936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cb935-df62-4a19-97c2-2780a67cf200 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.663307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966cb935-df62-4a19-97c2-2780a67cf200 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.677530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cb935-df62-4a19-97c2-2780a67cf200 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:36.679911 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966cb935-df62-4a19-97c2-2780a67cf200 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-966cb935-df62-4a19-97c2-2780a67cf200 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:36.705897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:36.706825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:36.707701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:36.707701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:36.707701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:36.707701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:36.707701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 321/634] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:36.712746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cb935-df62-4a19-97c2-2780a67cf200 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:36.737063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90738244-c9ec-43d1-8e2a-47c98604fc86 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.775063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8beb4-c05b-4315-94d9-390c1f48537c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.776131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8beb4-c05b-4315-94d9-390c1f48537c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.776624 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8beb4-c05b-4315-94d9-390c1f48537c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.777175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa8beb4-c05b-4315-94d9-390c1f48537c None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.778251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8beb4-c05b-4315-94d9-390c1f48537c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:36.778771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8beb4-c05b-4315-94d9-390c1f48537c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:36.780401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa8beb4-c05b-4315-94d9-390c1f48537c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:36.799043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 314/635] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 12 13:14:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:36.800236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8beb4-c05b-4315-94d9-390c1f48537c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:36.817940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:36.838466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966cb935-df62-4a19-97c2-2780a67cf200 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.884686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2294db9-03b3-4ff8-bf48-f7a10792a092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.885684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2294db9-03b3-4ff8-bf48-f7a10792a092 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.886183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2294db9-03b3-4ff8-bf48-f7a10792a092 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.886476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2294db9-03b3-4ff8-bf48-f7a10792a092 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.901727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:36.903249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2294db9-03b3-4ff8-bf48-f7a10792a092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c2294db9-03b3-4ff8-bf48-f7a10792a092 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:36.915138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:36.916117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:36.916973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 322/636] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:36.918194 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2294db9-03b3-4ff8-bf48-f7a10792a092 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:36.929713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa8beb4-c05b-4315-94d9-390c1f48537c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:36.977156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d440-32c7-452d-9f68-13f128faadd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:36.978096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d440-32c7-452d-9f68-13f128faadd5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:36.978381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d440-32c7-452d-9f68-13f128faadd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:36.978664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d440-32c7-452d-9f68-13f128faadd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:36.979470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3d440-32c7-452d-9f68-13f128faadd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:36.979779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3d440-32c7-452d-9f68-13f128faadd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:36.981257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da3d440-32c7-452d-9f68-13f128faadd5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:37.002071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 315/637] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:37.002071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d440-32c7-452d-9f68-13f128faadd5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:37.045820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2294db9-03b3-4ff8-bf48-f7a10792a092 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:37.076961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:37.095529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b36aa-3acb-4f4e-916c-ec61695181e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:37.096189 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b36aa-3acb-4f4e-916c-ec61695181e8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.096480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b36aa-3acb-4f4e-916c-ec61695181e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.096760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b36aa-3acb-4f4e-916c-ec61695181e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.123801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d440-32c7-452d-9f68-13f128faadd5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:37.169943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b36aa-3acb-4f4e-916c-ec61695181e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:37.170815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6c9b1-ae33-4483-801a-7d8752506b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:37.171487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b36aa-3acb-4f4e-916c-ec61695181e8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:37.172037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6c9b1-ae33-4483-801a-7d8752506b13 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.172521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6c9b1-ae33-4483-801a-7d8752506b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.173011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6c9b1-ae33-4483-801a-7d8752506b13 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.174954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6c9b1-ae33-4483-801a-7d8752506b13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:37.175480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6c9b1-ae33-4483-801a-7d8752506b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:37.177199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6c9b1-ae33-4483-801a-7d8752506b13 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:37.189524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 316/638] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:37.189686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:37.191163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6c9b1-ae33-4483-801a-7d8752506b13 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:37.301598 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 323/639] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:37.302517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b36aa-3acb-4f4e-916c-ec61695181e8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:37.318892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6c9b1-ae33-4483-801a-7d8752506b13 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:37.364219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:37.373587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069f240-977b-4134-8f5a-5b8a61fa9a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:37.375942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069f240-977b-4134-8f5a-5b8a61fa9a8e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.376384 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069f240-977b-4134-8f5a-5b8a61fa9a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.377104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069f240-977b-4134-8f5a-5b8a61fa9a8e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.386847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069f240-977b-4134-8f5a-5b8a61fa9a8e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:37.389361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5069f240-977b-4134-8f5a-5b8a61fa9a8e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:37.405281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-5069f240-977b-4134-8f5a-5b8a61fa9a8e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:37.412021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:37.413168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 12 13:14:37.414379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:37.416221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b36aa-3acb-4f4e-916c-ec61695181e8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:37.416845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 317/640] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:37.417854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069f240-977b-4134-8f5a-5b8a61fa9a8e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:37.461415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb7700-5fac-499b-8ceb-41bc08c46c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:37.462215 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb7700-5fac-499b-8ceb-41bc08c46c44 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.462496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb7700-5fac-499b-8ceb-41bc08c46c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.462767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb7700-5fac-499b-8ceb-41bc08c46c44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.474125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb7700-5fac-499b-8ceb-41bc08c46c44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:37.475844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb7700-5fac-499b-8ceb-41bc08c46c44 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-43bb7700-5fac-499b-8ceb-41bc08c46c44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:37.498019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:37.498953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:37.500448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:37.500448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:37.500448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:37.500448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:37.501277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 324/641] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:37.502141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb7700-5fac-499b-8ceb-41bc08c46c44 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:37.530393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069f240-977b-4134-8f5a-5b8a61fa9a8e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:37.567884 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc5f63-0041-49e5-bab6-36500674c56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:37.568689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5f63-0041-49e5-bab6-36500674c56a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.568981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5f63-0041-49e5-bab6-36500674c56a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.569253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc5f63-0041-49e5-bab6-36500674c56a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.580373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc5f63-0041-49e5-bab6-36500674c56a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:37.581855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc5f63-0041-49e5-bab6-36500674c56a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-40dc5f63-0041-49e5-bab6-36500674c56a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:37.600920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:37.601874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:37.602861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:37.602861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:37.602861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:37.602861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:37.602861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 318/642] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:37.603776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01688a83-09d3-47bd-812b-e8f76a0b5eba None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.603776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01688a83-09d3-47bd-812b-e8f76a0b5eba None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.603922 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01688a83-09d3-47bd-812b-e8f76a0b5eba None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.605273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 319/643] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:37.609854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01688a83-09d3-47bd-812b-e8f76a0b5eba None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:37.629460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb7700-5fac-499b-8ceb-41bc08c46c44 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:37.658088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:37.667787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6555619-9962-423d-96a5-d3d4a3409626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:37.668478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6555619-9962-423d-96a5-d3d4a3409626 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.668761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6555619-9962-423d-96a5-d3d4a3409626 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.669041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6555619-9962-423d-96a5-d3d4a3409626 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.669929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6555619-9962-423d-96a5-d3d4a3409626 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:37.670252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6555619-9962-423d-96a5-d3d4a3409626 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:37.672351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6555619-9962-423d-96a5-d3d4a3409626 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:37.690225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 325/644] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 12 13:14:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:37.691047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6555619-9962-423d-96a5-d3d4a3409626 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:37.695462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:37.724073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01688a83-09d3-47bd-812b-e8f76a0b5eba None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:37.759582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3797e3b7-1748-48fe-9995-c5a95c5014b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:37.760174 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797e3b7-1748-48fe-9995-c5a95c5014b6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.760451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797e3b7-1748-48fe-9995-c5a95c5014b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.760813 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3797e3b7-1748-48fe-9995-c5a95c5014b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.761579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797e3b7-1748-48fe-9995-c5a95c5014b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:37.761913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797e3b7-1748-48fe-9995-c5a95c5014b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:37.763272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3797e3b7-1748-48fe-9995-c5a95c5014b6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:37.775319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 320/645] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:37.776584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2ff15-d758-4920-816b-2067135e2c5d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.776877 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2ff15-d758-4920-816b-2067135e2c5d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.777149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf2ff15-d758-4920-816b-2067135e2c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.793591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-edf2ff15-d758-4920-816b-2067135e2c5d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:37.794765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-edf2ff15-d758-4920-816b-2067135e2c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:37.802162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6555619-9962-423d-96a5-d3d4a3409626 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:37.838308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec127790-6474-461a-988c-3e4164d6e2d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:37.839007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec127790-6474-461a-988c-3e4164d6e2d4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:37.839550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec127790-6474-461a-988c-3e4164d6e2d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:37.839550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec127790-6474-461a-988c-3e4164d6e2d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:37.840264 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec127790-6474-461a-988c-3e4164d6e2d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:37.840571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec127790-6474-461a-988c-3e4164d6e2d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:37.841965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec127790-6474-461a-988c-3e4164d6e2d4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:37.853890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 326/646] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:37.859099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec127790-6474-461a-988c-3e4164d6e2d4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:37.864899 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 321/647] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:37.865755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2ff15-d758-4920-816b-2067135e2c5d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:37.910778 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:37.926071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:37.966842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec127790-6474-461a-988c-3e4164d6e2d4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:37.987423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf2ff15-d758-4920-816b-2067135e2c5d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.000972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da33173-ae04-4b83-b3f4-c54519909c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.001658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33173-ae04-4b83-b3f4-c54519909c2b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.001945 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33173-ae04-4b83-b3f4-c54519909c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.002223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33173-ae04-4b83-b3f4-c54519909c2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.025224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020e2cd-dc0a-4d23-abbb-70646b1cfde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.025998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020e2cd-dc0a-4d23-abbb-70646b1cfde7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.026401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020e2cd-dc0a-4d23-abbb-70646b1cfde7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.026678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a020e2cd-dc0a-4d23-abbb-70646b1cfde7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.034410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020e2cd-dc0a-4d23-abbb-70646b1cfde7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:38.035846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a020e2cd-dc0a-4d23-abbb-70646b1cfde7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-a020e2cd-dc0a-4d23-abbb-70646b1cfde7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:38.051349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:38.052314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:38.053236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 322/648] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:38.054427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020e2cd-dc0a-4d23-abbb-70646b1cfde7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.077870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da33173-ae04-4b83-b3f4-c54519909c2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:38.079481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da33173-ae04-4b83-b3f4-c54519909c2b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:38.140738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:38.152988 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:38.165642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a020e2cd-dc0a-4d23-abbb-70646b1cfde7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.182190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 327/649] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:38.186620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da33173-ae04-4b83-b3f4-c54519909c2b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.202633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc2483-b159-44c5-a44e-59a97d5f77fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.203561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc2483-b159-44c5-a44e-59a97d5f77fb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.203845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc2483-b159-44c5-a44e-59a97d5f77fb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.204122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcc2483-b159-44c5-a44e-59a97d5f77fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.210904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc2483-b159-44c5-a44e-59a97d5f77fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:38.212346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcc2483-b159-44c5-a44e-59a97d5f77fb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3dcc2483-b159-44c5-a44e-59a97d5f77fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:38.227424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:38.228452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 323/650] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:38.229347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc2483-b159-44c5-a44e-59a97d5f77fb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.309638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da33173-ae04-4b83-b3f4-c54519909c2b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.337044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcc2483-b159-44c5-a44e-59a97d5f77fb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.344856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3de26-232a-4a57-9f72-90377363c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.345693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3de26-232a-4a57-9f72-90377363c4ca None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.345977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3de26-232a-4a57-9f72-90377363c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.346251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a3de26-232a-4a57-9f72-90377363c4ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.356418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a3de26-232a-4a57-9f72-90377363c4ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-86a3de26-232a-4a57-9f72-90377363c4ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:38.364908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:38.365881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:38.366703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 328/651] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:38.367378 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3de26-232a-4a57-9f72-90377363c4ca None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.375888 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50918f25-d9df-46d3-833e-b64328ca2d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.376779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50918f25-d9df-46d3-833e-b64328ca2d64 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.377235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50918f25-d9df-46d3-833e-b64328ca2d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.377691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50918f25-d9df-46d3-833e-b64328ca2d64 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.378678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50918f25-d9df-46d3-833e-b64328ca2d64 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:38.379140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50918f25-d9df-46d3-833e-b64328ca2d64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:38.380794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50918f25-d9df-46d3-833e-b64328ca2d64 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:38.396642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:38.400596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 324/652] 10.222.0.23 () {56 vars in 1137 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:38.401595 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50918f25-d9df-46d3-833e-b64328ca2d64 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.414882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:38.483182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a3de26-232a-4a57-9f72-90377363c4ca None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.515253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50918f25-d9df-46d3-833e-b64328ca2d64 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.516839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcf28e-aaaf-41b7-a8cc-783756eb3337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.517671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcf28e-aaaf-41b7-a8cc-783756eb3337 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.517954 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcf28e-aaaf-41b7-a8cc-783756eb3337 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.518228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcf28e-aaaf-41b7-a8cc-783756eb3337 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.528444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcf28e-aaaf-41b7-a8cc-783756eb3337 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:38.529940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdcf28e-aaaf-41b7-a8cc-783756eb3337 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-bbdcf28e-aaaf-41b7-a8cc-783756eb3337 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:38.548534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:38.549780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:38.550731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:38.550731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:38.550731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:38.550731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:38.550972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e655ba-2c7c-4a0e-9a67-a65d9e3b1644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.550972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e655ba-2c7c-4a0e-9a67-a65d9e3b1644 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.550972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e655ba-2c7c-4a0e-9a67-a65d9e3b1644 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.551189 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 329/653] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:38.551261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e655ba-2c7c-4a0e-9a67-a65d9e3b1644 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.551341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcf28e-aaaf-41b7-a8cc-783756eb3337 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.551415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e655ba-2c7c-4a0e-9a67-a65d9e3b1644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:38.551415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e655ba-2c7c-4a0e-9a67-a65d9e3b1644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:38.552622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e655ba-2c7c-4a0e-9a67-a65d9e3b1644 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:38.565225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 325/654] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:38.566123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e655ba-2c7c-4a0e-9a67-a65d9e3b1644 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.627832 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:38.636393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:38.655378 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcf28e-aaaf-41b7-a8cc-783756eb3337 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.669967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e655ba-2c7c-4a0e-9a67-a65d9e3b1644 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.696081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe591d1-1091-4f9d-be0f-c07bc29d4f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.696749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe591d1-1091-4f9d-be0f-c07bc29d4f4d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.697032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe591d1-1091-4f9d-be0f-c07bc29d4f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.697306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe591d1-1091-4f9d-be0f-c07bc29d4f4d None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.698080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe591d1-1091-4f9d-be0f-c07bc29d4f4d None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:38.698389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe591d1-1091-4f9d-be0f-c07bc29d4f4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:38.699762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe591d1-1091-4f9d-be0f-c07bc29d4f4d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:38.704618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d8bde-770f-43ec-b2c0-722b9d4f6ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.705291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d8bde-770f-43ec-b2c0-722b9d4f6ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.705607 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d8bde-770f-43ec-b2c0-722b9d4f6ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.705889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1d8bde-770f-43ec-b2c0-722b9d4f6ae2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.708254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 330/655] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:38.710420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6d75d-06c3-447e-98dc-d7df15703e1d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.710712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6d75d-06c3-447e-98dc-d7df15703e1d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.710986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6d75d-06c3-447e-98dc-d7df15703e1d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.712387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 331/656] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:38.712927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6d75d-06c3-447e-98dc-d7df15703e1d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.772679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d8bde-770f-43ec-b2c0-722b9d4f6ae2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:38.774163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1d8bde-770f-43ec-b2c0-722b9d4f6ae2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:38.812757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6d75d-06c3-447e-98dc-d7df15703e1d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.839932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:38.846429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4390525-4c78-4ead-8db0-419ea2f4f49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:38.847080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4390525-4c78-4ead-8db0-419ea2f4f49c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:38.847372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4390525-4c78-4ead-8db0-419ea2f4f49c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:38.847644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4390525-4c78-4ead-8db0-419ea2f4f49c None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:38.848368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4390525-4c78-4ead-8db0-419ea2f4f49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:38.848673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4390525-4c78-4ead-8db0-419ea2f4f49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:38.850061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4390525-4c78-4ead-8db0-419ea2f4f49c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:38.856766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:38.862746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 332/657] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:38.863552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4390525-4c78-4ead-8db0-419ea2f4f49c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.872766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 326/658] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:38.873619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d8bde-770f-43ec-b2c0-722b9d4f6ae2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:38.963807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4390525-4c78-4ead-8db0-419ea2f4f49c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:38.971282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1d8bde-770f-43ec-b2c0-722b9d4f6ae2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.001927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d7538-404f-4abc-a9f7-da05455d505b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.002581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7538-404f-4abc-a9f7-da05455d505b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.002862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7538-404f-4abc-a9f7-da05455d505b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.003135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7538-404f-4abc-a9f7-da05455d505b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.003962 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d7538-404f-4abc-a9f7-da05455d505b None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=1cc0050f14664630a36814ca670f124f) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:39.004274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d7538-404f-4abc-a9f7-da05455d505b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.005725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5d7538-404f-4abc-a9f7-da05455d505b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.010175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bea542-691b-46d4-a886-3862473c95dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.011119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bea542-691b-46d4-a886-3862473c95dd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.011404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bea542-691b-46d4-a886-3862473c95dd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.011679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bea542-691b-46d4-a886-3862473c95dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.012409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 333/659] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/role_assignments?user.id=1cc0050f14664630a36814ca670f124f&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:39.025803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1695-fb84-4c53-a06c-3edb4ce54abf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.025803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1695-fb84-4c53-a06c-3edb4ce54abf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.025803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67aa1695-fb84-4c53-a06c-3edb4ce54abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.028071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bea542-691b-46d4-a886-3862473c95dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.029723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bea542-691b-46d4-a886-3862473c95dd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.040186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-67aa1695-fb84-4c53-a06c-3edb4ce54abf None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:39.041899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-67aa1695-fb84-4c53-a06c-3edb4ce54abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b2bea542-691b-46d4-a886-3862473c95dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:39.052177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:39.053175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:39.054608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:39.054608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:39.054608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:39.054608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:39.057714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 327/660] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:39.058979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bea542-691b-46d4-a886-3862473c95dd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.089448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:39.111539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:39.129424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 334/661] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:39.129424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa1695-fb84-4c53-a06c-3edb4ce54abf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.193523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bea542-691b-46d4-a886-3862473c95dd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.242549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91698876-d0e5-40f0-b035-426143bc1d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.243792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91698876-d0e5-40f0-b035-426143bc1d7b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.244078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91698876-d0e5-40f0-b035-426143bc1d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.244357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91698876-d0e5-40f0-b035-426143bc1d7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.252504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91698876-d0e5-40f0-b035-426143bc1d7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.252997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67aa1695-fb84-4c53-a06c-3edb4ce54abf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.253757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91698876-d0e5-40f0-b035-426143bc1d7b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-91698876-d0e5-40f0-b035-426143bc1d7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:39.267833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:39.268769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:39.270009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:39.270009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:39.270009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:39.270009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:39.270009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:39.270009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:39.270009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:39.270009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:39.270009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 328/662] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:39.270627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91698876-d0e5-40f0-b035-426143bc1d7b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.291815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25865a1c-5d19-4367-ad98-c599fac8bcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.292768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25865a1c-5d19-4367-ad98-c599fac8bcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.293159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25865a1c-5d19-4367-ad98-c599fac8bcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.293581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25865a1c-5d19-4367-ad98-c599fac8bcb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.304480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25865a1c-5d19-4367-ad98-c599fac8bcb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.305976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25865a1c-5d19-4367-ad98-c599fac8bcb1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-25865a1c-5d19-4367-ad98-c599fac8bcb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:39.324910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:39.325892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:39.326770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:39.326770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:39.326770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:39.326770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:39.326770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 335/663] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:39.327142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25865a1c-5d19-4367-ad98-c599fac8bcb1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.369738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:39.378174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:39.390984 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91698876-d0e5-40f0-b035-426143bc1d7b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.427579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634746b-2f93-4d09-921c-936d3a840ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.428242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634746b-2f93-4d09-921c-936d3a840ffc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.428531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634746b-2f93-4d09-921c-936d3a840ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.428808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7634746b-2f93-4d09-921c-936d3a840ffc None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.429588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634746b-2f93-4d09-921c-936d3a840ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:39.429897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634746b-2f93-4d09-921c-936d3a840ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.431273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7634746b-2f93-4d09-921c-936d3a840ffc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.440521 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25865a1c-5d19-4367-ad98-c599fac8bcb1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.448762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 329/664] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:39.448762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634746b-2f93-4d09-921c-936d3a840ffc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.480354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2b5cd-aac0-482b-83e6-794170028e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.481155 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2b5cd-aac0-482b-83e6-794170028e5d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.481477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2b5cd-aac0-482b-83e6-794170028e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.481737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee2b5cd-aac0-482b-83e6-794170028e5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.492594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee2b5cd-aac0-482b-83e6-794170028e5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4ee2b5cd-aac0-482b-83e6-794170028e5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:39.501066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:39.502326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 336/665] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:39.503195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2b5cd-aac0-482b-83e6-794170028e5d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.557432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7634746b-2f93-4d09-921c-936d3a840ffc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.598300 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc9fcd-c2ee-4afd-9704-3dcfa5430129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.598955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc9fcd-c2ee-4afd-9704-3dcfa5430129 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.599241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc9fcd-c2ee-4afd-9704-3dcfa5430129 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.599516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc9fcd-c2ee-4afd-9704-3dcfa5430129 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.600306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc9fcd-c2ee-4afd-9704-3dcfa5430129 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:39.600613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc9fcd-c2ee-4afd-9704-3dcfa5430129 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.602069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abc9fcd-c2ee-4afd-9704-3dcfa5430129 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.619302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 330/666] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:39.619302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc9fcd-c2ee-4afd-9704-3dcfa5430129 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.619582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee2b5cd-aac0-482b-83e6-794170028e5d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.621848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:39.622163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:39.658437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc969db1-aca6-4014-84ce-bef1b3ea0149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.659159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc969db1-aca6-4014-84ce-bef1b3ea0149 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.659443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc969db1-aca6-4014-84ce-bef1b3ea0149 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.659721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc969db1-aca6-4014-84ce-bef1b3ea0149 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.660508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc969db1-aca6-4014-84ce-bef1b3ea0149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:39.660820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc969db1-aca6-4014-84ce-bef1b3ea0149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.662383 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc969db1-aca6-4014-84ce-bef1b3ea0149 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.675414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 337/667] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:39.676086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc969db1-aca6-4014-84ce-bef1b3ea0149 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.734578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc9fcd-c2ee-4afd-9704-3dcfa5430129 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.771202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132f89c-d9fc-415d-903c-3378dc303cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.772453 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f89c-d9fc-415d-903c-3378dc303cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.772874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f89c-d9fc-415d-903c-3378dc303cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.773440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5132f89c-d9fc-415d-903c-3378dc303cf2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.780039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc969db1-aca6-4014-84ce-bef1b3ea0149 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.783322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132f89c-d9fc-415d-903c-3378dc303cf2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.784772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5132f89c-d9fc-415d-903c-3378dc303cf2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-5132f89c-d9fc-415d-903c-3378dc303cf2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:39.802809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:39.803823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 12 13:14:39.804766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:39.806063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 331/668] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:39.806063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132f89c-d9fc-415d-903c-3378dc303cf2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.818802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5839-c59f-4356-bd9a-2db03c85a31a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.819482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5839-c59f-4356-bd9a-2db03c85a31a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.819775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5839-c59f-4356-bd9a-2db03c85a31a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.820050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca5839-c59f-4356-bd9a-2db03c85a31a None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.820797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca5839-c59f-4356-bd9a-2db03c85a31a None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:39.821105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca5839-c59f-4356-bd9a-2db03c85a31a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.822510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca5839-c59f-4356-bd9a-2db03c85a31a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.835868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 338/669] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:39.835868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5839-c59f-4356-bd9a-2db03c85a31a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.864670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:39.870475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:39.911221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5132f89c-d9fc-415d-903c-3378dc303cf2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.943564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca5839-c59f-4356-bd9a-2db03c85a31a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:39.946294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b19198-9e8d-465e-a24c-8b421e5e750a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.946948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b19198-9e8d-465e-a24c-8b421e5e750a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.947234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b19198-9e8d-465e-a24c-8b421e5e750a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.947512 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b19198-9e8d-465e-a24c-8b421e5e750a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:39.948348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b19198-9e8d-465e-a24c-8b421e5e750a None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=0ff10db6937b4f2b889c30dd5fad8111) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:39.948663 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b19198-9e8d-465e-a24c-8b421e5e750a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:39.950119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b19198-9e8d-465e-a24c-8b421e5e750a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:39.965428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 332/670] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/role_assignments?user.id=0ff10db6937b4f2b889c30dd5fad8111&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:39.965428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b19198-9e8d-465e-a24c-8b421e5e750a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:39.992368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738b5a9-2357-40ef-96d8-a749d6508168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:39.993431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738b5a9-2357-40ef-96d8-a749d6508168 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:39.993899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738b5a9-2357-40ef-96d8-a749d6508168 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:39.994187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2738b5a9-2357-40ef-96d8-a749d6508168 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:40.003214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738b5a9-2357-40ef-96d8-a749d6508168 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:40.004831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2738b5a9-2357-40ef-96d8-a749d6508168 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2738b5a9-2357-40ef-96d8-a749d6508168 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:40.021547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:40.022519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:40.023426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:40.023426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:40.023426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:40.023426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:40.023426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:40.023426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:40.023426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:40.023426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:40.023426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 339/671] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:40.024199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738b5a9-2357-40ef-96d8-a749d6508168 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:40.102404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b19198-9e8d-465e-a24c-8b421e5e750a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:40.127471 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:40.145405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:40.147273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2738b5a9-2357-40ef-96d8-a749d6508168 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:40.150873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e62e2f-33f8-47e3-a019-a18298508321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:40.151553 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e62e2f-33f8-47e3-a019-a18298508321 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:40.151843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e62e2f-33f8-47e3-a019-a18298508321 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:40.152124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e62e2f-33f8-47e3-a019-a18298508321 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:40.153010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e62e2f-33f8-47e3-a019-a18298508321 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:40.153327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e62e2f-33f8-47e3-a019-a18298508321 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:40.154808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e62e2f-33f8-47e3-a019-a18298508321 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:40.182160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 333/672] 10.222.0.23 () {56 vars in 1139 bytes} [Tue Jul 12 13:14:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:40.183041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e62e2f-33f8-47e3-a019-a18298508321 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:40.198725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960a9f5-fb72-45d2-8a1a-0f5d07eb4b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:40.200038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960a9f5-fb72-45d2-8a1a-0f5d07eb4b2f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:40.200315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960a9f5-fb72-45d2-8a1a-0f5d07eb4b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:40.200886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960a9f5-fb72-45d2-8a1a-0f5d07eb4b2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:40.215562 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960a9f5-fb72-45d2-8a1a-0f5d07eb4b2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:40.217025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960a9f5-fb72-45d2-8a1a-0f5d07eb4b2f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f960a9f5-fb72-45d2-8a1a-0f5d07eb4b2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:40.239663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:40.243476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:40.245335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:40.245335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:40.245335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:40.245335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:40.245335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 340/673] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:40.245335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960a9f5-fb72-45d2-8a1a-0f5d07eb4b2f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:40.318485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e62e2f-33f8-47e3-a019-a18298508321 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:40.357212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b3608-303b-4612-a601-6b3be7bd6e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:40.358064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b3608-303b-4612-a601-6b3be7bd6e06 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:40.358437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b3608-303b-4612-a601-6b3be7bd6e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:40.358827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6b3608-303b-4612-a601-6b3be7bd6e06 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:40.360092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b3608-303b-4612-a601-6b3be7bd6e06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:40.360705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b3608-303b-4612-a601-6b3be7bd6e06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:40.362574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6b3608-303b-4612-a601-6b3be7bd6e06 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:40.369877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960a9f5-fb72-45d2-8a1a-0f5d07eb4b2f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:40.379843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 334/674] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:40.380956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b3608-303b-4612-a601-6b3be7bd6e06 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:40.407967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0936c-052a-4bcf-a5da-9fc72011d4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:40.408950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0936c-052a-4bcf-a5da-9fc72011d4be None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:40.409451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0936c-052a-4bcf-a5da-9fc72011d4be None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:40.409927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc0936c-052a-4bcf-a5da-9fc72011d4be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:40.418700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:40.431020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:40.484235 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc0936c-052a-4bcf-a5da-9fc72011d4be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:40.486122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc0936c-052a-4bcf-a5da-9fc72011d4be None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:40.493254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6b3608-303b-4612-a601-6b3be7bd6e06 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:40.528582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c85ee0-e8d5-4a56-918d-08582482cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:40.529250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c85ee0-e8d5-4a56-918d-08582482cb29 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:40.529826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c85ee0-e8d5-4a56-918d-08582482cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:40.529939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c85ee0-e8d5-4a56-918d-08582482cb29 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:40.531265 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c85ee0-e8d5-4a56-918d-08582482cb29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:40.531584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c85ee0-e8d5-4a56-918d-08582482cb29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:40.532956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c85ee0-e8d5-4a56-918d-08582482cb29 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:40.548187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 335/675] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:40.549463 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c85ee0-e8d5-4a56-918d-08582482cb29 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:40.609167 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 341/676] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:40.610482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0936c-052a-4bcf-a5da-9fc72011d4be None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:40.658589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c85ee0-e8d5-4a56-918d-08582482cb29 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:40.663800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:40.666634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:40.695283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fdc70-0759-4190-82c8-d27a7637c8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:40.696091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fdc70-0759-4190-82c8-d27a7637c8cd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:40.696466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fdc70-0759-4190-82c8-d27a7637c8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:40.696839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fdc70-0759-4190-82c8-d27a7637c8cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:40.697822 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fdc70-0759-4190-82c8-d27a7637c8cd None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b4f12d08d7084387a1bea7865bf77d0e) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:40.698254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fdc70-0759-4190-82c8-d27a7637c8cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:40.699772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3fdc70-0759-4190-82c8-d27a7637c8cd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:40.713516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 336/677] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:40 2022] GET /identity/v3/role_assignments?user.id=b4f12d08d7084387a1bea7865bf77d0e&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:40.714703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fdc70-0759-4190-82c8-d27a7637c8cd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:40.754852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc0936c-052a-4bcf-a5da-9fc72011d4be None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:40.847271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac549335-3e23-43ba-bc6c-2e51821568db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:40.848081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac549335-3e23-43ba-bc6c-2e51821568db None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:40.848368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac549335-3e23-43ba-bc6c-2e51821568db None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:40.848648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac549335-3e23-43ba-bc6c-2e51821568db None admin] PATH_INFO: `/v3/users/swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:40.871545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac549335-3e23-43ba-bc6c-2e51821568db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:40.872337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac549335-3e23-43ba-bc6c-2e51821568db None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:40.873815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fdc70-0759-4190-82c8-d27a7637c8cd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ac549335-3e23-43ba-bc6c-2e51821568db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:40.890594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:40.891667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:40.893058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 342/678] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:40.898289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac549335-3e23-43ba-bc6c-2e51821568db None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:40.918312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5003d-e255-49b0-a9c2-3849ab9cbd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:40.919505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5003d-e255-49b0-a9c2-3849ab9cbd19 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:40.920008 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5003d-e255-49b0-a9c2-3849ab9cbd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:40.920532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5003d-e255-49b0-a9c2-3849ab9cbd19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:40.935052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5003d-e255-49b0-a9c2-3849ab9cbd19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:40.938465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5003d-e255-49b0-a9c2-3849ab9cbd19 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d6a5003d-e255-49b0-a9c2-3849ab9cbd19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:40.957769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:40.958748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:40.959632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:40.959632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:40.959632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:40.959632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:40.962083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 337/679] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:40.971000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5003d-e255-49b0-a9c2-3849ab9cbd19 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:41.003938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:41.026999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:41.032627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac549335-3e23-43ba-bc6c-2e51821568db None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:41.085115 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5f1a8-d21e-4362-8882-8885cc1f91c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:41.086170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5f1a8-d21e-4362-8882-8885cc1f91c9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.086666 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5f1a8-d21e-4362-8882-8885cc1f91c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.087201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e5f1a8-d21e-4362-8882-8885cc1f91c9 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.088360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5f1a8-d21e-4362-8882-8885cc1f91c9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:41.088918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5f1a8-d21e-4362-8882-8885cc1f91c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:41.091174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e5f1a8-d21e-4362-8882-8885cc1f91c9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:41.105849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5003d-e255-49b0-a9c2-3849ab9cbd19 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:41.113207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 343/680] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 12 13:14:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:41.119287 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5f1a8-d21e-4362-8882-8885cc1f91c9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:41.181641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1055c-a088-4859-a9bc-150286cf240b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:41.182323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1055c-a088-4859-a9bc-150286cf240b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.182614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1055c-a088-4859-a9bc-150286cf240b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.182904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d1055c-a088-4859-a9bc-150286cf240b None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.183640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1055c-a088-4859-a9bc-150286cf240b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:41.183970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1055c-a088-4859-a9bc-150286cf240b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:41.185402 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d1055c-a088-4859-a9bc-150286cf240b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:41.211575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 338/681] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:41.216894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1055c-a088-4859-a9bc-150286cf240b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:41.240863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e5f1a8-d21e-4362-8882-8885cc1f91c9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:41.289055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f93b4f4-2aa5-4881-a9d0-9990c5a54043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:41.290538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93b4f4-2aa5-4881-a9d0-9990c5a54043 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.291401 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93b4f4-2aa5-4881-a9d0-9990c5a54043 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.291883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f93b4f4-2aa5-4881-a9d0-9990c5a54043 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.317650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:41.326412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:41.342352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d1055c-a088-4859-a9bc-150286cf240b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:41.380452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0830f3-8456-4e25-b15a-d429419817db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:41.381357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93b4f4-2aa5-4881-a9d0-9990c5a54043 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:41.381755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0830f3-8456-4e25-b15a-d429419817db None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.382335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0830f3-8456-4e25-b15a-d429419817db None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.382761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0830f3-8456-4e25-b15a-d429419817db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.383245 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f93b4f4-2aa5-4881-a9d0-9990c5a54043 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:41.390589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0830f3-8456-4e25-b15a-d429419817db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:41.392413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0830f3-8456-4e25-b15a-d429419817db None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-de0830f3-8456-4e25-b15a-d429419817db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:41.409970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:41.410908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:41.411798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:41.411798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:41.411798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:41.411798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:41.411798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:41.411798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:41.411798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:41.411798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:41.414682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 339/682] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:41.416378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455c5f5-9068-47c9-8ddb-2b3b39777b7e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.416870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455c5f5-9068-47c9-8ddb-2b3b39777b7e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.417341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455c5f5-9068-47c9-8ddb-2b3b39777b7e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.419190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 340/683] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:41.420324 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455c5f5-9068-47c9-8ddb-2b3b39777b7e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:41.504632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 344/684] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:41.506390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e3bdf-2aa0-4fe2-988f-6b7980e486cd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.506913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e3bdf-2aa0-4fe2-988f-6b7980e486cd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.507396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11e3bdf-2aa0-4fe2-988f-6b7980e486cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.525450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f11e3bdf-2aa0-4fe2-988f-6b7980e486cd None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:41.527166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f11e3bdf-2aa0-4fe2-988f-6b7980e486cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:41.542393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455c5f5-9068-47c9-8ddb-2b3b39777b7e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:41.577924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1fff0-1a3f-482f-bdf3-e7dd1bdae89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:41.578600 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1fff0-1a3f-482f-bdf3-e7dd1bdae89c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.578925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1fff0-1a3f-482f-bdf3-e7dd1bdae89c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.579139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef1fff0-1a3f-482f-bdf3-e7dd1bdae89c None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.579941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1fff0-1a3f-482f-bdf3-e7dd1bdae89c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:41.580224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1fff0-1a3f-482f-bdf3-e7dd1bdae89c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:41.581672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef1fff0-1a3f-482f-bdf3-e7dd1bdae89c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:41.587157 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:41.590163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:41.594913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 341/685] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:41.595725 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1fff0-1a3f-482f-bdf3-e7dd1bdae89c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:41.610547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 345/686] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:41.611869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e3bdf-2aa0-4fe2-988f-6b7980e486cd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:41.698784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef1fff0-1a3f-482f-bdf3-e7dd1bdae89c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:41.724419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11e3bdf-2aa0-4fe2-988f-6b7980e486cd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:41.734211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551c4df-2806-4733-8862-8d24a70dca5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:41.735012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551c4df-2806-4733-8862-8d24a70dca5d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.735566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551c4df-2806-4733-8862-8d24a70dca5d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.735566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551c4df-2806-4733-8862-8d24a70dca5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.746471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551c4df-2806-4733-8862-8d24a70dca5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:41.747918 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551c4df-2806-4733-8862-8d24a70dca5d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:41.762535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaaadf-758b-4936-b646-991b9863c17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:41.763592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaaadf-758b-4936-b646-991b9863c17a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.764087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaaadf-758b-4936-b646-991b9863c17a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.764625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcaaadf-758b-4936-b646-991b9863c17a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d551c4df-2806-4733-8862-8d24a70dca5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:41.766273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:41.767134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:41.768027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:41.768027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:41.768027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:41.768027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:41.769201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcaaadf-758b-4936-b646-991b9863c17a None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=83837e4f8f9b4e538658876cfb56db16) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:41.769762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcaaadf-758b-4936-b646-991b9863c17a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:41.771489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dcaaadf-758b-4936-b646-991b9863c17a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:41.772145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 342/687] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:41.772822 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551c4df-2806-4733-8862-8d24a70dca5d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:41.779650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 346/688] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:41 2022] GET /identity/v3/role_assignments?user.id=83837e4f8f9b4e538658876cfb56db16&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:41.781416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaaadf-758b-4936-b646-991b9863c17a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:41.848774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:41.853767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:41.901757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcaaadf-758b-4936-b646-991b9863c17a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:41.944537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca7048-a7a9-45b0-a581-8479551c93c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:41.945639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca7048-a7a9-45b0-a581-8479551c93c0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:41.946037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca7048-a7a9-45b0-a581-8479551c93c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:41.946424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dca7048-a7a9-45b0-a581-8479551c93c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:41.947282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca7048-a7a9-45b0-a581-8479551c93c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:41.947700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca7048-a7a9-45b0-a581-8479551c93c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:41.949266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dca7048-a7a9-45b0-a581-8479551c93c0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:41.963677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 347/689] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:41.973228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca7048-a7a9-45b0-a581-8479551c93c0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:42.016499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551c4df-2806-4733-8862-8d24a70dca5d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:42.058677 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fb6c9-472e-4fc0-a76d-6bedc6630e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:42.059601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fb6c9-472e-4fc0-a76d-6bedc6630e61 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.059987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fb6c9-472e-4fc0-a76d-6bedc6630e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.060363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2fb6c9-472e-4fc0-a76d-6bedc6630e61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.072365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2fb6c9-472e-4fc0-a76d-6bedc6630e61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-cc2fb6c9-472e-4fc0-a76d-6bedc6630e61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:42.080499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:42.081505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:42.082485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:42.084642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 343/690] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:42.091688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fb6c9-472e-4fc0-a76d-6bedc6630e61 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:42.105737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dca7048-a7a9-45b0-a581-8479551c93c0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:42.121740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:42.155500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e762c-38e8-4e70-acba-ee4f1eb572ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:42.156324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e762c-38e8-4e70-acba-ee4f1eb572ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.156612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e762c-38e8-4e70-acba-ee4f1eb572ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.156889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e762c-38e8-4e70-acba-ee4f1eb572ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.165200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e762c-38e8-4e70-acba-ee4f1eb572ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:42.166711 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e762c-38e8-4e70-acba-ee4f1eb572ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-fd4e762c-38e8-4e70-acba-ee4f1eb572ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:42.187843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:42.188983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:42.190087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:42.190087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:42.190087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:42.190087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:42.190087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:42.190087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:42.190087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:42.190087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:42.193422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 348/691] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:42.196640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e762c-38e8-4e70-acba-ee4f1eb572ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:42.229669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2fb6c9-472e-4fc0-a76d-6bedc6630e61 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:42.248732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:42.276543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886053e-53b5-499a-a8a6-fa9051656144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:42.277500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886053e-53b5-499a-a8a6-fa9051656144 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.277801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886053e-53b5-499a-a8a6-fa9051656144 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.278070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886053e-53b5-499a-a8a6-fa9051656144 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.279230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886053e-53b5-499a-a8a6-fa9051656144 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:42.279802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886053e-53b5-499a-a8a6-fa9051656144 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:42.281836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c886053e-53b5-499a-a8a6-fa9051656144 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:42.301482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 344/692] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:42.307435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e03638-368b-48bd-952f-6313c8da72c8 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.307730 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e03638-368b-48bd-952f-6313c8da72c8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.308016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e03638-368b-48bd-952f-6313c8da72c8 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.313401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 345/693] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:42.313401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e03638-368b-48bd-952f-6313c8da72c8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:42.337123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e762c-38e8-4e70-acba-ee4f1eb572ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:42.373990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964d8ba7-e435-4334-bdc2-7c7d655e53ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:42.375448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d8ba7-e435-4334-bdc2-7c7d655e53ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.375896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d8ba7-e435-4334-bdc2-7c7d655e53ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.376335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964d8ba7-e435-4334-bdc2-7c7d655e53ec None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.377282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d8ba7-e435-4334-bdc2-7c7d655e53ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:42.377764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d8ba7-e435-4334-bdc2-7c7d655e53ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:42.379475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964d8ba7-e435-4334-bdc2-7c7d655e53ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:42.391987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 349/694] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:42.393656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d792a8d-b82e-455f-9e9c-c97fe682aa20 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.394227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d792a8d-b82e-455f-9e9c-c97fe682aa20 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.394711 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d792a8d-b82e-455f-9e9c-c97fe682aa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.412967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8d792a8d-b82e-455f-9e9c-c97fe682aa20 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:42.414742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8d792a8d-b82e-455f-9e9c-c97fe682aa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:42.439440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e03638-368b-48bd-952f-6313c8da72c8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:42.443501 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:42.490082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f59180-55c5-4248-bcd8-b99871fbf3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:42.490879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f59180-55c5-4248-bcd8-b99871fbf3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.491164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f59180-55c5-4248-bcd8-b99871fbf3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.491630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f59180-55c5-4248-bcd8-b99871fbf3e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.497617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 350/695] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:42.498787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d792a8d-b82e-455f-9e9c-c97fe682aa20 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:42.504471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f59180-55c5-4248-bcd8-b99871fbf3e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:42.506969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f59180-55c5-4248-bcd8-b99871fbf3e8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-08f59180-55c5-4248-bcd8-b99871fbf3e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:42.527615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:42.528541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:42.529467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:42.529467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:42.529467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:42.529467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:42.529467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 346/696] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:42.535743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f59180-55c5-4248-bcd8-b99871fbf3e8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:42.578179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:42.635737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d792a8d-b82e-455f-9e9c-c97fe682aa20 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:42.658589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f59180-55c5-4248-bcd8-b99871fbf3e8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:42.677334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44feca-6bd1-4e93-b052-30547324b4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:42.678323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44feca-6bd1-4e93-b052-30547324b4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.678883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44feca-6bd1-4e93-b052-30547324b4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.679473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d44feca-6bd1-4e93-b052-30547324b4a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.680531 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44feca-6bd1-4e93-b052-30547324b4a1 None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b69264fe173045e08c282171cd7370af) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:42.681047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44feca-6bd1-4e93-b052-30547324b4a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:42.682861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d44feca-6bd1-4e93-b052-30547324b4a1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:42.690262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 351/697] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:42 2022] GET /identity/v3/role_assignments?user.id=b69264fe173045e08c282171cd7370af&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:42.691320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44feca-6bd1-4e93-b052-30547324b4a1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:42.701092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042e640-145f-4988-ad84-56588cf0c102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:42.702811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042e640-145f-4988-ad84-56588cf0c102 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.703313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042e640-145f-4988-ad84-56588cf0c102 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.703885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e042e640-145f-4988-ad84-56588cf0c102 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.704894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042e640-145f-4988-ad84-56588cf0c102 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:42.705463 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042e640-145f-4988-ad84-56588cf0c102 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:42.712196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e042e640-145f-4988-ad84-56588cf0c102 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:42.744179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 347/698] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:42.744469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:42.751626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042e640-145f-4988-ad84-56588cf0c102 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:42.859579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:42.880632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e042e640-145f-4988-ad84-56588cf0c102 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:42.904493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d44feca-6bd1-4e93-b052-30547324b4a1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:42.924073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3baaa1-bc79-45ee-8181-8600fc851c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:42.925181 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3baaa1-bc79-45ee-8181-8600fc851c22 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.925648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3baaa1-bc79-45ee-8181-8600fc851c22 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.926093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3baaa1-bc79-45ee-8181-8600fc851c22 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.933703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3baaa1-bc79-45ee-8181-8600fc851c22 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:42.935323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3baaa1-bc79-45ee-8181-8600fc851c22 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:42.948345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f910b9-5d2f-4463-bec8-11e496043906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:42.948518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f910b9-5d2f-4463-bec8-11e496043906 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:42.948798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f910b9-5d2f-4463-bec8-11e496043906 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:42.949074 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f910b9-5d2f-4463-bec8-11e496043906 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-af3baaa1-bc79-45ee-8181-8600fc851c22 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:42.953229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:42.954370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:42.955520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:42.957632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 348/699] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 12 13:14:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:42.961471 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f910b9-5d2f-4463-bec8-11e496043906 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:42.961471 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f910b9-5d2f-4463-bec8-11e496043906 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:42.963411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3baaa1-bc79-45ee-8181-8600fc851c22 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-73f910b9-5d2f-4463-bec8-11e496043906 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:42.984886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:42.986073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:42.986968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:42.986968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:42.986968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:42.986968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:42.986968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 352/700] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:42.992495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f910b9-5d2f-4463-bec8-11e496043906 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:43.086628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:43.106960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3baaa1-bc79-45ee-8181-8600fc851c22 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:43.117551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f910b9-5d2f-4463-bec8-11e496043906 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:43.153290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a911e-7cbb-445b-bb5e-db89621a2966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:43.154471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a911e-7cbb-445b-bb5e-db89621a2966 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:43.154471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a911e-7cbb-445b-bb5e-db89621a2966 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:43.154748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8a911e-7cbb-445b-bb5e-db89621a2966 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:43.156426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a911e-7cbb-445b-bb5e-db89621a2966 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:43.156755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a911e-7cbb-445b-bb5e-db89621a2966 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:43.158220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8a911e-7cbb-445b-bb5e-db89621a2966 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:43.161183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795863e-e1c8-4541-bf86-fb980ea30d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:43.162164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795863e-e1c8-4541-bf86-fb980ea30d61 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:43.162639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795863e-e1c8-4541-bf86-fb980ea30d61 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:43.163059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5795863e-e1c8-4541-bf86-fb980ea30d61 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:43.163942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795863e-e1c8-4541-bf86-fb980ea30d61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:43.164397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795863e-e1c8-4541-bf86-fb980ea30d61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:43.165968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5795863e-e1c8-4541-bf86-fb980ea30d61 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:43.178780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 349/701] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 12 13:14:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:43.183610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 353/702] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:43.185293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a911e-7cbb-445b-bb5e-db89621a2966 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:43.189144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795863e-e1c8-4541-bf86-fb980ea30d61 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:43.191747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:43.318616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5795863e-e1c8-4541-bf86-fb980ea30d61 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:43.339605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8a911e-7cbb-445b-bb5e-db89621a2966 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:43.358372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab55ea64-e6b4-409d-a51d-c13b2d7ff744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:43.359678 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55ea64-e6b4-409d-a51d-c13b2d7ff744 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:43.360349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55ea64-e6b4-409d-a51d-c13b2d7ff744 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:43.361235 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab55ea64-e6b4-409d-a51d-c13b2d7ff744 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:43.362482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55ea64-e6b4-409d-a51d-c13b2d7ff744 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:43.364586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab55ea64-e6b4-409d-a51d-c13b2d7ff744 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:43.373718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:43.387489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7794a-cbbd-4630-8fa9-25ff60c2ddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:43.388529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7794a-cbbd-4630-8fa9-25ff60c2ddc9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:43.388964 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7794a-cbbd-4630-8fa9-25ff60c2ddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:43.389471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add7794a-cbbd-4630-8fa9-25ff60c2ddc9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:43.391785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 354/703] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:14:43 2022] POST /identity/v3/users => generated 254 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:43.393395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebfc16-b8b3-40ee-b4b5-118cba1d8da6 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:43.393829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebfc16-b8b3-40ee-b4b5-118cba1d8da6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:43.394205 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebfc16-b8b3-40ee-b4b5-118cba1d8da6 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:43.396119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 355/704] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:43.401188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53842-df55-40f9-a338-729901633aaf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:43.401660 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53842-df55-40f9-a338-729901633aaf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:43.402002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be53842-df55-40f9-a338-729901633aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:43.437803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5be53842-df55-40f9-a338-729901633aaf None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:43.439091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5be53842-df55-40f9-a338-729901633aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:43.477309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7794a-cbbd-4630-8fa9-25ff60c2ddc9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:43.479517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add7794a-cbbd-4630-8fa9-25ff60c2ddc9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:43.489409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:43.597252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 356/705] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:43.616475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 350/706] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 12 13:14:43.621890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be53842-df55-40f9-a338-729901633aaf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:43.627569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7794a-cbbd-4630-8fa9-25ff60c2ddc9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:43.750560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be53842-df55-40f9-a338-729901633aaf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:43.758889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add7794a-cbbd-4630-8fa9-25ff60c2ddc9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:43.762098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:43.783574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:43.799646 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bafd4-d8b8-4179-85ae-bfeb0ac8a518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:43.800115 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2603944-33d4-48fa-a4a3-4022585f7ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:43.800917 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bafd4-d8b8-4179-85ae-bfeb0ac8a518 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:43.801237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2603944-33d4-48fa-a4a3-4022585f7ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:43.801411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bafd4-d8b8-4179-85ae-bfeb0ac8a518 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:43.801801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02bafd4-d8b8-4179-85ae-bfeb0ac8a518 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:43.801991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2603944-33d4-48fa-a4a3-4022585f7ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:43.802407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2603944-33d4-48fa-a4a3-4022585f7ff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:43.802739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bafd4-d8b8-4179-85ae-bfeb0ac8a518 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:43.804203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02bafd4-d8b8-4179-85ae-bfeb0ac8a518 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:43.815083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2603944-33d4-48fa-a4a3-4022585f7ff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:43.816849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2603944-33d4-48fa-a4a3-4022585f7ff9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b02bafd4-d8b8-4179-85ae-bfeb0ac8a518 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:43.821500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:43.822708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:14:43.823762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:43.825980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 351/707] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Jul 12 13:14:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:43.835553 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bafd4-d8b8-4179-85ae-bfeb0ac8a518 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f2603944-33d4-48fa-a4a3-4022585f7ff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:43.845006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:43.845991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:43.847055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:43.847055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:43.847055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:43.847055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:43.848697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 357/708] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:43.854162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2603944-33d4-48fa-a4a3-4022585f7ff9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:43.962526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02bafd4-d8b8-4179-85ae-bfeb0ac8a518 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:43.977925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2603944-33d4-48fa-a4a3-4022585f7ff9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:44.001591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04429218-19ff-4455-8fd5-8a3407a7e498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:44.002349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429218-19ff-4455-8fd5-8a3407a7e498 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.002798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429218-19ff-4455-8fd5-8a3407a7e498 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.003178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04429218-19ff-4455-8fd5-8a3407a7e498 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.004001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04429218-19ff-4455-8fd5-8a3407a7e498 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:44.004412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04429218-19ff-4455-8fd5-8a3407a7e498 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:44.006003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04429218-19ff-4455-8fd5-8a3407a7e498 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:44.012187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 352/709] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 12 13:14:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:44.020067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04429218-19ff-4455-8fd5-8a3407a7e498 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:44.029875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5602-6f51-45ea-976e-4d2dd1f2ad89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:44.031485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5602-6f51-45ea-976e-4d2dd1f2ad89 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.031927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5602-6f51-45ea-976e-4d2dd1f2ad89 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.032355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6d5602-6f51-45ea-976e-4d2dd1f2ad89 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.033742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5602-6f51-45ea-976e-4d2dd1f2ad89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:44.034207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5602-6f51-45ea-976e-4d2dd1f2ad89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:44.036403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6d5602-6f51-45ea-976e-4d2dd1f2ad89 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:44.051648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 358/710] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:44.057488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5602-6f51-45ea-976e-4d2dd1f2ad89 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:44.073864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:44.126521 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:44.212369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04429218-19ff-4455-8fd5-8a3407a7e498 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:44.237952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6d5602-6f51-45ea-976e-4d2dd1f2ad89 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:44.263954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66104c07-1dbf-4cfa-9c40-00969e585f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:44.264619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66104c07-1dbf-4cfa-9c40-00969e585f02 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.264954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66104c07-1dbf-4cfa-9c40-00969e585f02 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.265224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66104c07-1dbf-4cfa-9c40-00969e585f02 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.266058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66104c07-1dbf-4cfa-9c40-00969e585f02 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:44.266363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66104c07-1dbf-4cfa-9c40-00969e585f02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:44.267720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66104c07-1dbf-4cfa-9c40-00969e585f02 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:44.275799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 353/711] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 12 13:14:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:44.277401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eca22-afee-4359-9eb0-ce3b3ce3dcbe None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.277925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eca22-afee-4359-9eb0-ce3b3ce3dcbe None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.278406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6eca22-afee-4359-9eb0-ce3b3ce3dcbe None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.280907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 354/712] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:44.293414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89cd30-6496-49bc-8c84-594f00a68e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:44.295270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69452-b3b6-452a-977a-6a770cf1f2c1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.295786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69452-b3b6-452a-977a-6a770cf1f2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.296990 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace69452-b3b6-452a-977a-6a770cf1f2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.307745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89cd30-6496-49bc-8c84-594f00a68e10 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.308386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89cd30-6496-49bc-8c84-594f00a68e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.308868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a89cd30-6496-49bc-8c84-594f00a68e10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.321239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89cd30-6496-49bc-8c84-594f00a68e10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:44.323147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a89cd30-6496-49bc-8c84-594f00a68e10 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-9a89cd30-6496-49bc-8c84-594f00a68e10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:44.340828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:44.341850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:44.343380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:44.343380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:44.343380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:44.343380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:44.343380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:44.343380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:44.343380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:44.343380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:44.343825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ace69452-b3b6-452a-977a-6a770cf1f2c1 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:44.343825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ace69452-b3b6-452a-977a-6a770cf1f2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:44.346553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 359/713] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:44.352762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89cd30-6496-49bc-8c84-594f00a68e10 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:44.439945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 355/714] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:44.462126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace69452-b3b6-452a-977a-6a770cf1f2c1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:44.504871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:44.555053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:44.569447 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a89cd30-6496-49bc-8c84-594f00a68e10 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:44.581035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace69452-b3b6-452a-977a-6a770cf1f2c1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:44.617234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bcffb-dd34-4b82-b329-dace54f8fae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:44.618221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bcffb-dd34-4b82-b329-dace54f8fae0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.618602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bcffb-dd34-4b82-b329-dace54f8fae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.618978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5bcffb-dd34-4b82-b329-dace54f8fae0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.619738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bcffb-dd34-4b82-b329-dace54f8fae0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:44.621203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5bcffb-dd34-4b82-b329-dace54f8fae0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-0b5bcffb-dd34-4b82-b329-dace54f8fae0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:44.634113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:44.635208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:44.636061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 356/715] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:44 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:44.645683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d8ae4-926e-4ea8-9197-f280af367b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:44.646680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d8ae4-926e-4ea8-9197-f280af367b5c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.647165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d8ae4-926e-4ea8-9197-f280af367b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.647642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d8ae4-926e-4ea8-9197-f280af367b5c None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.648685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d8ae4-926e-4ea8-9197-f280af367b5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:44.649215 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d8ae4-926e-4ea8-9197-f280af367b5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:44.650277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bcffb-dd34-4b82-b329-dace54f8fae0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:44.651325 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d8ae4-926e-4ea8-9197-f280af367b5c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:44.664675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 360/716] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:44.670730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d8ae4-926e-4ea8-9197-f280af367b5c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:44.839267 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:44.842630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d8ae4-926e-4ea8-9197-f280af367b5c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:44.869538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5bcffb-dd34-4b82-b329-dace54f8fae0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:44.916589 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf312e3-7b50-4db7-8eb8-1075279e54d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:44.918321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf312e3-7b50-4db7-8eb8-1075279e54d5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.918873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf312e3-7b50-4db7-8eb8-1075279e54d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.919940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf312e3-7b50-4db7-8eb8-1075279e54d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.921519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf312e3-7b50-4db7-8eb8-1075279e54d5 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=1cc0050f14664630a36814ca670f124f) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:44.922463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf312e3-7b50-4db7-8eb8-1075279e54d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:44.925140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf312e3-7b50-4db7-8eb8-1075279e54d5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:44.935043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77facad-2f65-4d1f-a4c1-172115f2d2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:44.936248 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77facad-2f65-4d1f-a4c1-172115f2d2ad None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:44.939209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77facad-2f65-4d1f-a4c1-172115f2d2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:44.939684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77facad-2f65-4d1f-a4c1-172115f2d2ad None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:44.940583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77facad-2f65-4d1f-a4c1-172115f2d2ad None admin] RBAC: Adding query filter params (name=network) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:44.941098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77facad-2f65-4d1f-a4c1-172115f2d2ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:44.949409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77facad-2f65-4d1f-a4c1-172115f2d2ad None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:44.950114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 361/717] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:44 2022] GET /identity/v3/role_assignments?user.id=1cc0050f14664630a36814ca670f124f&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:44.956260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 357/718] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:14:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:44.961973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf312e3-7b50-4db7-8eb8-1075279e54d5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:44.989848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:45.073599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf312e3-7b50-4db7-8eb8-1075279e54d5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:45.112108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e6ab13-cc0f-49df-a673-45fabb5f77a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:45.113060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6ab13-cc0f-49df-a673-45fabb5f77a9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:45.113555 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6ab13-cc0f-49df-a673-45fabb5f77a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:45.114033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e6ab13-cc0f-49df-a673-45fabb5f77a9 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:45.114925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6ab13-cc0f-49df-a673-45fabb5f77a9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:45.115411 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6ab13-cc0f-49df-a673-45fabb5f77a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:45.116952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e6ab13-cc0f-49df-a673-45fabb5f77a9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:45.123720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 362/719] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:14:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:45.414188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce703ae0-5052-4841-8e8f-c6bd11e5e65c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:45.414670 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce703ae0-5052-4841-8e8f-c6bd11e5e65c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:45.414670 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce703ae0-5052-4841-8e8f-c6bd11e5e65c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:45.416303 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 358/720] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:45.422169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f774af-49d2-49f1-82a8-f9ac42fac519 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:45.422869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f774af-49d2-49f1-82a8-f9ac42fac519 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:45.423377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f774af-49d2-49f1-82a8-f9ac42fac519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:45.451408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f0f774af-49d2-49f1-82a8-f9ac42fac519 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:45.453072 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f0f774af-49d2-49f1-82a8-f9ac42fac519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:45.559981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 363/721] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:45.606655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce703ae0-5052-4841-8e8f-c6bd11e5e65c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:45.730190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce703ae0-5052-4841-8e8f-c6bd11e5e65c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:45.765743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48a905-d3b4-4702-b0bd-08f68d4f69ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:45.766545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48a905-d3b4-4702-b0bd-08f68d4f69ef None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:45.766826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48a905-d3b4-4702-b0bd-08f68d4f69ef None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:45.767094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48a905-d3b4-4702-b0bd-08f68d4f69ef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:45.777651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48a905-d3b4-4702-b0bd-08f68d4f69ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-2f48a905-d3b4-4702-b0bd-08f68d4f69ef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:45.786124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:45.787042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:45.788014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 359/722] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:45.795159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f774af-49d2-49f1-82a8-f9ac42fac519 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:45.820427 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:45.931733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f774af-49d2-49f1-82a8-f9ac42fac519 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:45.971398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db7d21-2914-4fb7-9291-69e52be18dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:45.972084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db7d21-2914-4fb7-9291-69e52be18dcc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:45.972373 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db7d21-2914-4fb7-9291-69e52be18dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:45.972656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68db7d21-2914-4fb7-9291-69e52be18dcc None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:45.973432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db7d21-2914-4fb7-9291-69e52be18dcc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:45.973746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db7d21-2914-4fb7-9291-69e52be18dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:45.975123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68db7d21-2914-4fb7-9291-69e52be18dcc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:45.991827 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 364/723] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:45.997143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48a905-d3b4-4702-b0bd-08f68d4f69ef None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:46.023421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:46.142307 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48a905-d3b4-4702-b0bd-08f68d4f69ef None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:46.190553 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb0c0a-b242-47af-ac50-e52537d88865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:46.191344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb0c0a-b242-47af-ac50-e52537d88865 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:46.191625 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb0c0a-b242-47af-ac50-e52537d88865 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:46.191899 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb0c0a-b242-47af-ac50-e52537d88865 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:46.205585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb0c0a-b242-47af-ac50-e52537d88865 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:46.207075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb0c0a-b242-47af-ac50-e52537d88865 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9adb0c0a-b242-47af-ac50-e52537d88865 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:46.233232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:46.234219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:46.235190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:46.235190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:46.235190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:46.235190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:46.235190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 360/724] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:46.248516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db7d21-2914-4fb7-9291-69e52be18dcc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:46.379429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:46.415627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68db7d21-2914-4fb7-9291-69e52be18dcc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:46.460720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4e131-f458-4d9a-b5a4-5707781c8a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:46.461449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4e131-f458-4d9a-b5a4-5707781c8a71 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:46.461742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4e131-f458-4d9a-b5a4-5707781c8a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:46.462032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4e131-f458-4d9a-b5a4-5707781c8a71 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:46.462786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4e131-f458-4d9a-b5a4-5707781c8a71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:46.463101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4e131-f458-4d9a-b5a4-5707781c8a71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:46.464541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4e131-f458-4d9a-b5a4-5707781c8a71 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:46.483397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 365/725] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:46.484994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb0c0a-b242-47af-ac50-e52537d88865 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:46.558738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caa222-33b8-49bf-aed4-9db7bbf6e7e9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:46.559348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caa222-33b8-49bf-aed4-9db7bbf6e7e9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:46.559813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caa222-33b8-49bf-aed4-9db7bbf6e7e9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:46.562500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 366/726] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:46.568036 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a920c6-c3ed-4fb9-8fad-dc79c9db0566 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:46.568503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a920c6-c3ed-4fb9-8fad-dc79c9db0566 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:46.568930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a920c6-c3ed-4fb9-8fad-dc79c9db0566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:46.570310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:46.599039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-92a920c6-c3ed-4fb9-8fad-dc79c9db0566 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:46.600254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-92a920c6-c3ed-4fb9-8fad-dc79c9db0566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:46.606059 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb0c0a-b242-47af-ac50-e52537d88865 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:46.664033 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff9a8c-81d1-48e0-8ad5-2aa18dfd3be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:46.665137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff9a8c-81d1-48e0-8ad5-2aa18dfd3be0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:46.665595 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff9a8c-81d1-48e0-8ad5-2aa18dfd3be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:46.667095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ff9a8c-81d1-48e0-8ad5-2aa18dfd3be0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:46.681160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff9a8c-81d1-48e0-8ad5-2aa18dfd3be0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:46.682866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ff9a8c-81d1-48e0-8ad5-2aa18dfd3be0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d2ff9a8c-81d1-48e0-8ad5-2aa18dfd3be0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:46.708310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:46.709525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:46.710662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:46.714181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 367/727] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 12 13:14:46.719075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 361/728] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:46.721136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a920c6-c3ed-4fb9-8fad-dc79c9db0566 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:46.738094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff9a8c-81d1-48e0-8ad5-2aa18dfd3be0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:46.857691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ff9a8c-81d1-48e0-8ad5-2aa18dfd3be0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:46.859097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:46.890812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:46.903270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075861c-ca85-4cde-9ab0-b5ed7d4607bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:46.904878 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075861c-ca85-4cde-9ab0-b5ed7d4607bf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:46.905422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075861c-ca85-4cde-9ab0-b5ed7d4607bf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:46.906034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075861c-ca85-4cde-9ab0-b5ed7d4607bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:46.907560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a920c6-c3ed-4fb9-8fad-dc79c9db0566 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:46.919531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075861c-ca85-4cde-9ab0-b5ed7d4607bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f075861c-ca85-4cde-9ab0-b5ed7d4607bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:46.930183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:46.931348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:46.932410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:46.934925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 362/729] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:46.935975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a491e69-1a09-4a79-9624-5bee6cbc2bbf None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:46.936551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a491e69-1a09-4a79-9624-5bee6cbc2bbf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:46.939074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a491e69-1a09-4a79-9624-5bee6cbc2bbf None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:46.940935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 363/730] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:46.946425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a491e69-1a09-4a79-9624-5bee6cbc2bbf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:46.960352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadab374-bab8-4ccc-bb92-63479b0caa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:46.961040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadab374-bab8-4ccc-bb92-63479b0caa8f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:46.961331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadab374-bab8-4ccc-bb92-63479b0caa8f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:46.961727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadab374-bab8-4ccc-bb92-63479b0caa8f None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:46.963201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadab374-bab8-4ccc-bb92-63479b0caa8f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:46.963520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadab374-bab8-4ccc-bb92-63479b0caa8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:46.965099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadab374-bab8-4ccc-bb92-63479b0caa8f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:46.985928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 368/731] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 12 13:14:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:46.987319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efae3f-d036-4371-af80-5d2d0c76846c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:46.987621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efae3f-d036-4371-af80-5d2d0c76846c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:46.987907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efae3f-d036-4371-af80-5d2d0c76846c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.009540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-84efae3f-d036-4371-af80-5d2d0c76846c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:47.010811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-84efae3f-d036-4371-af80-5d2d0c76846c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:47.089424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a491e69-1a09-4a79-9624-5bee6cbc2bbf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:47.101689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 369/732] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:47.102822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efae3f-d036-4371-af80-5d2d0c76846c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:47.129208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86210f1c-fbf8-430b-8f3c-6e4522508515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:47.130633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86210f1c-fbf8-430b-8f3c-6e4522508515 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.131243 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86210f1c-fbf8-430b-8f3c-6e4522508515 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.131729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86210f1c-fbf8-430b-8f3c-6e4522508515 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.132827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86210f1c-fbf8-430b-8f3c-6e4522508515 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:47.133368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86210f1c-fbf8-430b-8f3c-6e4522508515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:47.135146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86210f1c-fbf8-430b-8f3c-6e4522508515 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:47.147528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 364/733] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:47.149645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86210f1c-fbf8-430b-8f3c-6e4522508515 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:47.178967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:47.212795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:47.252506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efae3f-d036-4371-af80-5d2d0c76846c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:47.290510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86210f1c-fbf8-430b-8f3c-6e4522508515 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:47.300647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d6ccc-0ccc-4bcf-b79c-f06b134b8fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:47.301064 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d6ccc-0ccc-4bcf-b79c-f06b134b8fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.301357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d6ccc-0ccc-4bcf-b79c-f06b134b8fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.301694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d6ccc-0ccc-4bcf-b79c-f06b134b8fd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.335445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b93f62-c731-4d3e-bcc7-d049df4c2039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:47.336125 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b93f62-c731-4d3e-bcc7-d049df4c2039 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.336414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b93f62-c731-4d3e-bcc7-d049df4c2039 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.336694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b93f62-c731-4d3e-bcc7-d049df4c2039 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.337742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b93f62-c731-4d3e-bcc7-d049df4c2039 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:47.339184 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b93f62-c731-4d3e-bcc7-d049df4c2039 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:47.361828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 365/734] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:14:47 2022] POST /identity/v3/services => generated 251 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:47.363000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b93f62-c731-4d3e-bcc7-d049df4c2039 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:47.398855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d6ccc-0ccc-4bcf-b79c-f06b134b8fd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:47.400722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d6ccc-0ccc-4bcf-b79c-f06b134b8fd9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:47.493176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b93f62-c731-4d3e-bcc7-d049df4c2039 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:47.501023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:47.503162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:47.542082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a65c9f-190f-4e1b-af9a-c4b1bb115b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:47.542903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a65c9f-190f-4e1b-af9a-c4b1bb115b6d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.543189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a65c9f-190f-4e1b-af9a-c4b1bb115b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.543465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a65c9f-190f-4e1b-af9a-c4b1bb115b6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.554524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a65c9f-190f-4e1b-af9a-c4b1bb115b6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:47.556013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a65c9f-190f-4e1b-af9a-c4b1bb115b6d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:47.562653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 370/735] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 12 13:14:47.564272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cf112-90ae-4195-b9ee-83d10c3c3a56 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.564721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cf112-90ae-4195-b9ee-83d10c3c3a56 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.564996 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72cf112-90ae-4195-b9ee-83d10c3c3a56 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.566595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 371/736] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:47.571658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d1609-8c1f-4602-895b-26796307a151 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.571948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d1609-8c1f-4602-895b-26796307a151 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.572227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d1609-8c1f-4602-895b-26796307a151 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.573764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 372/737] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:47.574348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d1609-8c1f-4602-895b-26796307a151 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-97a65c9f-190f-4e1b-af9a-c4b1bb115b6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:47.584417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:47.585515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:47.586408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:47.586408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:47.586408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:47.586408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:47.589547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 366/738] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:47.590952 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b7c6-9b53-4b3b-8060-99766b577928 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.591240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b7c6-9b53-4b3b-8060-99766b577928 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.591513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307b7c6-9b53-4b3b-8060-99766b577928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.609539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5307b7c6-9b53-4b3b-8060-99766b577928 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:47.610185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5307b7c6-9b53-4b3b-8060-99766b577928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:47.690530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 367/739] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:47.690530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205becdb-beea-463d-a321-0cb03413ad71 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.690530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205becdb-beea-463d-a321-0cb03413ad71 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.690530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205becdb-beea-463d-a321-0cb03413ad71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.700648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d1609-8c1f-4602-895b-26796307a151 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:47.704470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-205becdb-beea-463d-a321-0cb03413ad71 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:47.705717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-205becdb-beea-463d-a321-0cb03413ad71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:47.739627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720e8bf-1b66-4865-be0d-2d9663f786e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:47.740556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720e8bf-1b66-4865-be0d-2d9663f786e7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.740843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720e8bf-1b66-4865-be0d-2d9663f786e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.741147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720e8bf-1b66-4865-be0d-2d9663f786e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.751711 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720e8bf-1b66-4865-be0d-2d9663f786e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:47.753072 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720e8bf-1b66-4865-be0d-2d9663f786e7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-1720e8bf-1b66-4865-be0d-2d9663f786e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:47.774269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:47.775190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:47.783153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:47.783153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:47.783153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:47.783153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:47.783153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 373/740] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:47.783436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 368/741] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 12 13:14:47.783519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720e8bf-1b66-4865-be0d-2d9663f786e7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:47.783588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205becdb-beea-463d-a321-0cb03413ad71 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:47.783655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:47.798646 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:47.887459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205becdb-beea-463d-a321-0cb03413ad71 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:47.900480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720e8bf-1b66-4865-be0d-2d9663f786e7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:47.923051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18e7b7-7bce-4a04-a315-eef03f138c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:47.923714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18e7b7-7bce-4a04-a315-eef03f138c98 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.923998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18e7b7-7bce-4a04-a315-eef03f138c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.924275 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18e7b7-7bce-4a04-a315-eef03f138c98 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.924994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18e7b7-7bce-4a04-a315-eef03f138c98 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:47.926408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18e7b7-7bce-4a04-a315-eef03f138c98 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:47.938904 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13338fb-25e7-4901-87b0-2b5738b5c5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:47.939816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13338fb-25e7-4901-87b0-2b5738b5c5b5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:47.940109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13338fb-25e7-4901-87b0-2b5738b5c5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:47.940397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13338fb-25e7-4901-87b0-2b5738b5c5b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:47.941187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13338fb-25e7-4901-87b0-2b5738b5c5b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:47.941499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13338fb-25e7-4901-87b0-2b5738b5c5b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:47.942874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13338fb-25e7-4901-87b0-2b5738b5c5b5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:47.944630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 369/742] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:14:47 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:47.945689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18e7b7-7bce-4a04-a315-eef03f138c98 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:47.960651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 374/743] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:47.960651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13338fb-25e7-4901-87b0-2b5738b5c5b5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.031223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:48.073784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:48.076509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13338fb-25e7-4901-87b0-2b5738b5c5b5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.090981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18e7b7-7bce-4a04-a315-eef03f138c98 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.115304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4898563-8480-4d97-8f92-69ee4bb2dce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.116377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4898563-8480-4d97-8f92-69ee4bb2dce3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.116810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4898563-8480-4d97-8f92-69ee4bb2dce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.117271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4898563-8480-4d97-8f92-69ee4bb2dce3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.132757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4898563-8480-4d97-8f92-69ee4bb2dce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:48.136788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c7d1e-c5a4-4fe7-8ea5-8b5bb56df756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.139087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c7d1e-c5a4-4fe7-8ea5-8b5bb56df756 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.139087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c7d1e-c5a4-4fe7-8ea5-8b5bb56df756 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.139087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25c7d1e-c5a4-4fe7-8ea5-8b5bb56df756 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.139087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c7d1e-c5a4-4fe7-8ea5-8b5bb56df756 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:48.139392 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c7d1e-c5a4-4fe7-8ea5-8b5bb56df756 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:48.140599 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25c7d1e-c5a4-4fe7-8ea5-8b5bb56df756 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b4898563-8480-4d97-8f92-69ee4bb2dce3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:48.140838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:48.141811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:48.142649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:48.143275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 375/744] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:48.144126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4898563-8480-4d97-8f92-69ee4bb2dce3 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.153349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 370/745] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:48.154291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c7d1e-c5a4-4fe7-8ea5-8b5bb56df756 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.274410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25c7d1e-c5a4-4fe7-8ea5-8b5bb56df756 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.279414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4898563-8480-4d97-8f92-69ee4bb2dce3 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.304699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:48.306782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:48.312299 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405e30b-fc25-4690-b9a9-b4d49bad2179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.312979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405e30b-fc25-4690-b9a9-b4d49bad2179 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.313564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405e30b-fc25-4690-b9a9-b4d49bad2179 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.313851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405e30b-fc25-4690-b9a9-b4d49bad2179 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.314779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405e30b-fc25-4690-b9a9-b4d49bad2179 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:48.315093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405e30b-fc25-4690-b9a9-b4d49bad2179 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:48.317058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405e30b-fc25-4690-b9a9-b4d49bad2179 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:48.320300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28182991-5ac0-4aac-bae0-5829690dcca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.321117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182991-5ac0-4aac-bae0-5829690dcca2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.321425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182991-5ac0-4aac-bae0-5829690dcca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.321717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28182991-5ac0-4aac-bae0-5829690dcca2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.326706 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 371/746] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:48.327575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405e30b-fc25-4690-b9a9-b4d49bad2179 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.330314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28182991-5ac0-4aac-bae0-5829690dcca2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:48.332342 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28182991-5ac0-4aac-bae0-5829690dcca2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-28182991-5ac0-4aac-bae0-5829690dcca2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:48.349165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:48.351349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 376/747] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:14:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:48.353668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28182991-5ac0-4aac-bae0-5829690dcca2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.450564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405e30b-fc25-4690-b9a9-b4d49bad2179 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.463059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28182991-5ac0-4aac-bae0-5829690dcca2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.496710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21641d-52fb-48c9-905f-6688d511d3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.497875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21641d-52fb-48c9-905f-6688d511d3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.498107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21641d-52fb-48c9-905f-6688d511d3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.498387 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c21641d-52fb-48c9-905f-6688d511d3e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.502425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad51199-6ea8-4e8c-80d9-741d82fddc71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.503307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad51199-6ea8-4e8c-80d9-741d82fddc71 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.503668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad51199-6ea8-4e8c-80d9-741d82fddc71 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.504016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad51199-6ea8-4e8c-80d9-741d82fddc71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.506384 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21641d-52fb-48c9-905f-6688d511d3e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:48.508155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c21641d-52fb-48c9-905f-6688d511d3e1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:48.515850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad51199-6ea8-4e8c-80d9-741d82fddc71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:48.517403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad51199-6ea8-4e8c-80d9-741d82fddc71 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-5c21641d-52fb-48c9-905f-6688d511d3e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:48.524927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:48.526023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:48.527109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:48.527109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:48.527109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:48.527109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:48.527109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:48.527109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:48.527109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:48.527109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:48.529538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 372/748] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:48.530424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21641d-52fb-48c9-905f-6688d511d3e1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0ad51199-6ea8-4e8c-80d9-741d82fddc71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:48.541002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:48.542778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:48.544601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:48.544601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:48.544601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:48.544601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:48.548966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 377/749] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:48.548966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad51199-6ea8-4e8c-80d9-741d82fddc71 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.588139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:48.597788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:48.650366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c21641d-52fb-48c9-905f-6688d511d3e1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.660407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad51199-6ea8-4e8c-80d9-741d82fddc71 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.688971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18141663-c171-4ec9-9538-f5d91d65628e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.689697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141663-c171-4ec9-9538-f5d91d65628e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.690076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141663-c171-4ec9-9538-f5d91d65628e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.690261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18141663-c171-4ec9-9538-f5d91d65628e None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.691115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18141663-c171-4ec9-9538-f5d91d65628e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:48.691428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18141663-c171-4ec9-9538-f5d91d65628e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:48.693069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18141663-c171-4ec9-9538-f5d91d65628e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:48.699185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a7391-8a44-4139-adb1-3a14c6c38f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.700102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a7391-8a44-4139-adb1-3a14c6c38f14 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.700575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a7391-8a44-4139-adb1-3a14c6c38f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.701044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417a7391-8a44-4139-adb1-3a14c6c38f14 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.702074 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a7391-8a44-4139-adb1-3a14c6c38f14 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:48.703099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a7391-8a44-4139-adb1-3a14c6c38f14 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:48.704811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417a7391-8a44-4139-adb1-3a14c6c38f14 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:48.712779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 373/750] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 12 13:14:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:48.713405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18141663-c171-4ec9-9538-f5d91d65628e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.718762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 378/751] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:48.719847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a7391-8a44-4139-adb1-3a14c6c38f14 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.819227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:48.825668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417a7391-8a44-4139-adb1-3a14c6c38f14 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.826607 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18141663-c171-4ec9-9538-f5d91d65628e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:48.860110 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:48.863180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98080d9-e95f-414d-ab6b-591648e0873c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.864074 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98080d9-e95f-414d-ab6b-591648e0873c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.865073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98080d9-e95f-414d-ab6b-591648e0873c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.865885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98080d9-e95f-414d-ab6b-591648e0873c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.867695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f4ffe-3361-4f5f-9a1b-49a02b89760c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:48.868548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f4ffe-3361-4f5f-9a1b-49a02b89760c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:48.869445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f4ffe-3361-4f5f-9a1b-49a02b89760c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:48.869933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f4ffe-3361-4f5f-9a1b-49a02b89760c None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:48.870847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f4ffe-3361-4f5f-9a1b-49a02b89760c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:48.871339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f4ffe-3361-4f5f-9a1b-49a02b89760c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:48.873868 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f4ffe-3361-4f5f-9a1b-49a02b89760c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:48.893119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 374/752] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:48.894586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f4ffe-3361-4f5f-9a1b-49a02b89760c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:48.956199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98080d9-e95f-414d-ab6b-591648e0873c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:48.957835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f98080d9-e95f-414d-ab6b-591648e0873c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:49.035395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f4ffe-3361-4f5f-9a1b-49a02b89760c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:49.057366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:49.066515 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 379/753] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:14:49.067529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98080d9-e95f-414d-ab6b-591648e0873c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:49.087404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c3079-f0fb-49fe-946c-a34e29d9e113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:49.088699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c3079-f0fb-49fe-946c-a34e29d9e113 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:49.089174 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c3079-f0fb-49fe-946c-a34e29d9e113 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:49.089679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c3079-f0fb-49fe-946c-a34e29d9e113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:49.090724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c3079-f0fb-49fe-946c-a34e29d9e113 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b69264fe173045e08c282171cd7370af) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:49.091833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c3079-f0fb-49fe-946c-a34e29d9e113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:49.094489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6c3079-f0fb-49fe-946c-a34e29d9e113 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:49.103704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 375/754] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:48 2022] GET /identity/v3/role_assignments?user.id=b69264fe173045e08c282171cd7370af&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:49.104984 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c3079-f0fb-49fe-946c-a34e29d9e113 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:49.201768 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:49.243805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98080d9-e95f-414d-ab6b-591648e0873c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:49.290493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991388e1-42ff-422c-8349-3c523aeebecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:49.291302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991388e1-42ff-422c-8349-3c523aeebecf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:49.291586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991388e1-42ff-422c-8349-3c523aeebecf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:49.291862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991388e1-42ff-422c-8349-3c523aeebecf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:49.308023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991388e1-42ff-422c-8349-3c523aeebecf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:49.309818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991388e1-42ff-422c-8349-3c523aeebecf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-991388e1-42ff-422c-8349-3c523aeebecf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:49.333049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:49.334122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:49.335092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:49.346104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 380/755] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 12 13:14:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:49.354351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991388e1-42ff-422c-8349-3c523aeebecf None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:49.356310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6c3079-f0fb-49fe-946c-a34e29d9e113 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:49.401409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6ad52-0a0c-40f0-83bf-95036d1081b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:49.402228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6ad52-0a0c-40f0-83bf-95036d1081b0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:49.402516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6ad52-0a0c-40f0-83bf-95036d1081b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:49.402796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def6ad52-0a0c-40f0-83bf-95036d1081b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:49.416968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6ad52-0a0c-40f0-83bf-95036d1081b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:49.418586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def6ad52-0a0c-40f0-83bf-95036d1081b0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-def6ad52-0a0c-40f0-83bf-95036d1081b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:49.447631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:49.448572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:49.449567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:49.449567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:49.449567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:49.449567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:49.449567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 376/756] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:49.455750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6ad52-0a0c-40f0-83bf-95036d1081b0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:49.482657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:49.597191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991388e1-42ff-422c-8349-3c523aeebecf None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:49.666445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:49.675738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ad24c2-5218-4205-83f8-e09d1fbd1c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:49.676424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad24c2-5218-4205-83f8-e09d1fbd1c08 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:49.676712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad24c2-5218-4205-83f8-e09d1fbd1c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:49.676988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ad24c2-5218-4205-83f8-e09d1fbd1c08 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:49.677891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad24c2-5218-4205-83f8-e09d1fbd1c08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:49.678279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad24c2-5218-4205-83f8-e09d1fbd1c08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:49.680343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ad24c2-5218-4205-83f8-e09d1fbd1c08 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:49.699923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def6ad52-0a0c-40f0-83bf-95036d1081b0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:49.709896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 381/757] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 12 13:14:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:49.717433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb21aa-b44f-4b98-9c62-1610dcc14362 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:49.717433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb21aa-b44f-4b98-9c62-1610dcc14362 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:49.717433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edb21aa-b44f-4b98-9c62-1610dcc14362 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:49.717433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 382/758] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:49.717433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb21aa-b44f-4b98-9c62-1610dcc14362 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:49.746671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a249a6e-6118-42df-b1ac-3aa5ad6d22e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:49.747748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a249a6e-6118-42df-b1ac-3aa5ad6d22e8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:49.748245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a249a6e-6118-42df-b1ac-3aa5ad6d22e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:49.748715 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a249a6e-6118-42df-b1ac-3aa5ad6d22e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:49.750582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a249a6e-6118-42df-b1ac-3aa5ad6d22e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:49.751117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a249a6e-6118-42df-b1ac-3aa5ad6d22e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:49.752709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a249a6e-6118-42df-b1ac-3aa5ad6d22e8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:49.769291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 377/759] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:49.771111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec8d4-1f02-4816-9be3-2208bc684807 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:49.771627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec8d4-1f02-4816-9be3-2208bc684807 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:49.772210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678ec8d4-1f02-4816-9be3-2208bc684807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:49.792661 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-678ec8d4-1f02-4816-9be3-2208bc684807 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:49.794424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-678ec8d4-1f02-4816-9be3-2208bc684807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:49.857718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edb21aa-b44f-4b98-9c62-1610dcc14362 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:49.882564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:49.891984 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 378/760] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:14:49.893290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec8d4-1f02-4816-9be3-2208bc684807 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:49.913468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f80be-f845-415d-b70a-e5d8a73c73e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:49.914733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f80be-f845-415d-b70a-e5d8a73c73e1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:49.915120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f80be-f845-415d-b70a-e5d8a73c73e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:49.915497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706f80be-f845-415d-b70a-e5d8a73c73e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:49.929645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f80be-f845-415d-b70a-e5d8a73c73e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:49.931691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706f80be-f845-415d-b70a-e5d8a73c73e1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-706f80be-f845-415d-b70a-e5d8a73c73e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:49.955095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:49.956094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:49.956965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:49.956965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:49.956965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:49.956965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:49.961259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 383/761] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:49.962172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f80be-f845-415d-b70a-e5d8a73c73e1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:49.997897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:50.027757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678ec8d4-1f02-4816-9be3-2208bc684807 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:50.068916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba9dbb-4baf-4bec-a47c-9ec461e205ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:50.071374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba9dbb-4baf-4bec-a47c-9ec461e205ab None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.071882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba9dbb-4baf-4bec-a47c-9ec461e205ab None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.072372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba9dbb-4baf-4bec-a47c-9ec461e205ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.081350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba9dbb-4baf-4bec-a47c-9ec461e205ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:50.083543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba9dbb-4baf-4bec-a47c-9ec461e205ab None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-adba9dbb-4baf-4bec-a47c-9ec461e205ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:50.099196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:50.100214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:50.101901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:50.101901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:50.101901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:50.101901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:50.101901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:50.101901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:50.101901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:50.101901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:50.104594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 379/762] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:50.105770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba9dbb-4baf-4bec-a47c-9ec461e205ab None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:50.110606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706f80be-f845-415d-b70a-e5d8a73c73e1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:50.170762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fa384-c5fa-4578-a980-3cce4eecf853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:50.172217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fa384-c5fa-4578-a980-3cce4eecf853 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.173005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fa384-c5fa-4578-a980-3cce4eecf853 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.173286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fa384-c5fa-4578-a980-3cce4eecf853 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.174850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669fa384-c5fa-4578-a980-3cce4eecf853 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:50.178249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669fa384-c5fa-4578-a980-3cce4eecf853 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-669fa384-c5fa-4578-a980-3cce4eecf853 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:50.199701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:50.200745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:50.202021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 384/763] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Jul 12 13:14:49 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:50.203231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fa384-c5fa-4578-a980-3cce4eecf853 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:50.244176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba9dbb-4baf-4bec-a47c-9ec461e205ab None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:50.248776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:50.291676 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a472db-3a4c-4584-b772-0e94a336663a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:50.292611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a472db-3a4c-4584-b772-0e94a336663a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.292897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a472db-3a4c-4584-b772-0e94a336663a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.293175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a472db-3a4c-4584-b772-0e94a336663a None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.293949 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a472db-3a4c-4584-b772-0e94a336663a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:50.294256 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a472db-3a4c-4584-b772-0e94a336663a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:50.295630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a472db-3a4c-4584-b772-0e94a336663a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:50.308077 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:50.319269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 380/764] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:50.320297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a472db-3a4c-4584-b772-0e94a336663a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:50.334001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fa384-c5fa-4578-a980-3cce4eecf853 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:50.371503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a248a9d-38b7-435c-883d-f902e81d3d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:50.372080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248a9d-38b7-435c-883d-f902e81d3d6f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.372365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248a9d-38b7-435c-883d-f902e81d3d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.372644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248a9d-38b7-435c-883d-f902e81d3d6f None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.373463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a248a9d-38b7-435c-883d-f902e81d3d6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:50.373773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a248a9d-38b7-435c-883d-f902e81d3d6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:50.376025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a248a9d-38b7-435c-883d-f902e81d3d6f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:50.394510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 385/765] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:50.394510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b52b5-aa39-4356-8d82-e4e23564c530 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.394510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b52b5-aa39-4356-8d82-e4e23564c530 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.394510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b52b5-aa39-4356-8d82-e4e23564c530 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.394510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 386/766] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:50.394510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b52b5-aa39-4356-8d82-e4e23564c530 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:50.442640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a472db-3a4c-4584-b772-0e94a336663a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:50.482993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7094a29-71fd-4bce-8f46-f4a9ee60813c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:50.484375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7094a29-71fd-4bce-8f46-f4a9ee60813c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.484661 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7094a29-71fd-4bce-8f46-f4a9ee60813c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.484937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7094a29-71fd-4bce-8f46-f4a9ee60813c None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.485674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7094a29-71fd-4bce-8f46-f4a9ee60813c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:50.485976 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7094a29-71fd-4bce-8f46-f4a9ee60813c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:50.487367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7094a29-71fd-4bce-8f46-f4a9ee60813c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:50.490946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:50.491942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 381/767] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 12 13:14:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:50.492833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7094a29-71fd-4bce-8f46-f4a9ee60813c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:50.507601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b52b5-aa39-4356-8d82-e4e23564c530 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:50.543657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ed3dcb-95a5-417a-9523-1f2eb12fc06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:50.544530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed3dcb-95a5-417a-9523-1f2eb12fc06a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.544817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed3dcb-95a5-417a-9523-1f2eb12fc06a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.545091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed3dcb-95a5-417a-9523-1f2eb12fc06a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.552483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed3dcb-95a5-417a-9523-1f2eb12fc06a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:50.553995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ed3dcb-95a5-417a-9523-1f2eb12fc06a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:50.562349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-76ed3dcb-95a5-417a-9523-1f2eb12fc06a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:50.569037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:50.569930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:50.570803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:50.570803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:50.570803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:50.570803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:50.570803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:50.570803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:50.570803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:50.570803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:50.570803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 387/768] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:50.571844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308457ec-0c7f-4a8f-92bc-973cb2f5f84b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.571844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308457ec-0c7f-4a8f-92bc-973cb2f5f84b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.572000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308457ec-0c7f-4a8f-92bc-973cb2f5f84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.588839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-308457ec-0c7f-4a8f-92bc-973cb2f5f84b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:50.590072 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-308457ec-0c7f-4a8f-92bc-973cb2f5f84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:50.609137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7094a29-71fd-4bce-8f46-f4a9ee60813c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:50.645546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de3824-5bc8-403a-a1ac-7223c002834f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:50.646141 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de3824-5bc8-403a-a1ac-7223c002834f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.646422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de3824-5bc8-403a-a1ac-7223c002834f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.646698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9de3824-5bc8-403a-a1ac-7223c002834f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.647539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de3824-5bc8-403a-a1ac-7223c002834f None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b26fa3bc9c0d4e22a8abc5a4d98c9995) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:50.647869 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de3824-5bc8-403a-a1ac-7223c002834f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:50.649269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9de3824-5bc8-403a-a1ac-7223c002834f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:50.669397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 388/769] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:14:50.669397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308457ec-0c7f-4a8f-92bc-973cb2f5f84b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:50.669709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 382/770] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:50 2022] GET /identity/v3/role_assignments?user.id=b26fa3bc9c0d4e22a8abc5a4d98c9995&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:50.669709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de3824-5bc8-403a-a1ac-7223c002834f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:50.785477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:50.808361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308457ec-0c7f-4a8f-92bc-973cb2f5f84b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:50.812724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9de3824-5bc8-403a-a1ac-7223c002834f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:50.845612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:50.849858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444adaa-7e13-4497-8660-1c05e08db6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:50.851218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444adaa-7e13-4497-8660-1c05e08db6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.851600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444adaa-7e13-4497-8660-1c05e08db6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.852084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444adaa-7e13-4497-8660-1c05e08db6d8 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.852869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444adaa-7e13-4497-8660-1c05e08db6d8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:50.853244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444adaa-7e13-4497-8660-1c05e08db6d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:50.854755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444adaa-7e13-4497-8660-1c05e08db6d8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:50.858041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffad337-1771-428e-ac2a-1b2504122ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:50.859467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffad337-1771-428e-ac2a-1b2504122ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:50.860121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffad337-1771-428e-ac2a-1b2504122ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:50.860575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffad337-1771-428e-ac2a-1b2504122ae8 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:50.865551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 389/771] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 12 13:14:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:50.865551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444adaa-7e13-4497-8660-1c05e08db6d8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:50.866161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffad337-1771-428e-ac2a-1b2504122ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:50.866283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffad337-1771-428e-ac2a-1b2504122ae8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:50.868171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffad337-1771-428e-ac2a-1b2504122ae8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:50.881611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 383/772] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:50.882305 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffad337-1771-428e-ac2a-1b2504122ae8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:50.993332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444adaa-7e13-4497-8660-1c05e08db6d8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:51.002921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffad337-1771-428e-ac2a-1b2504122ae8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:51.033509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f48a61-0478-4a21-a90d-57b61672d2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:51.034450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48a61-0478-4a21-a90d-57b61672d2ae None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.034699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48a61-0478-4a21-a90d-57b61672d2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.034978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f48a61-0478-4a21-a90d-57b61672d2ae None admin] PATH_INFO: `/v3/services/network` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.035708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f48a61-0478-4a21-a90d-57b61672d2ae None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:51.037130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f48a61-0478-4a21-a90d-57b61672d2ae None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-98f48a61-0478-4a21-a90d-57b61672d2ae None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:51.051070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:51.052113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:14:51.052950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:51.054951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67645940-82e3-4ce0-a6f7-0fcffff1472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:51.055848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67645940-82e3-4ce0-a6f7-0fcffff1472b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.056368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67645940-82e3-4ce0-a6f7-0fcffff1472b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.056901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67645940-82e3-4ce0-a6f7-0fcffff1472b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.057116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 390/773] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:50 2022] GET /identity/v3/services/network => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:51.057986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f48a61-0478-4a21-a90d-57b61672d2ae None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:51.058472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67645940-82e3-4ce0-a6f7-0fcffff1472b None admin] RBAC: Adding query filter params (interface=public, service_id=ec728f69dd324bb0a3d9d4aa50c11ea4, region_id=RegionOne) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:51.058990 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67645940-82e3-4ce0-a6f7-0fcffff1472b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:51.060871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67645940-82e3-4ce0-a6f7-0fcffff1472b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:51.071047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:51.074176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 384/774] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 12 13:14:50 2022] GET /identity/v3/endpoints?service_id=ec728f69dd324bb0a3d9d4aa50c11ea4&interface=public®ion_id=RegionOne => generated 194 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:51.075223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67645940-82e3-4ce0-a6f7-0fcffff1472b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:51.146218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:51.195738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f48a61-0478-4a21-a90d-57b61672d2ae None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:51.210483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67645940-82e3-4ce0-a6f7-0fcffff1472b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:51.247004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc956c-c810-4d68-91c9-b87bd5c991d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:51.247920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc956c-c810-4d68-91c9-b87bd5c991d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.248275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc956c-c810-4d68-91c9-b87bd5c991d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.248724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddc956c-c810-4d68-91c9-b87bd5c991d4 None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/1cc0050f14664630a36814ca670f124f/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.251767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc956c-c810-4d68-91c9-b87bd5c991d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=1cc0050f14664630a36814ca670f124f, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:51.253626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddc956c-c810-4d68-91c9-b87bd5c991d4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:51.256709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e929e2a-fea8-4ee0-b699-cc32da93a4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:51.257680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e929e2a-fea8-4ee0-b699-cc32da93a4eb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.258247 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e929e2a-fea8-4ee0-b699-cc32da93a4eb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.258637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e929e2a-fea8-4ee0-b699-cc32da93a4eb None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.259409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e929e2a-fea8-4ee0-b699-cc32da93a4eb None admin] RBAC: Adding query filter params (name=network) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:51.259716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e929e2a-fea8-4ee0-b699-cc32da93a4eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:51.261717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e929e2a-fea8-4ee0-b699-cc32da93a4eb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:51.271442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 385/775] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:14:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:51.271442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e929e2a-fea8-4ee0-b699-cc32da93a4eb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:51.276324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 391/776] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:14:51 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/1cc0050f14664630a36814ca670f124f/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:51.277562 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc956c-c810-4d68-91c9-b87bd5c991d4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:51.369037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:51.388328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e929e2a-fea8-4ee0-b699-cc32da93a4eb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:51.405064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:51.410419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddc956c-c810-4d68-91c9-b87bd5c991d4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:51.430628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664555f9-b199-4bc2-9eb6-37974ae12c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:51.431754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664555f9-b199-4bc2-9eb6-37974ae12c27 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.432244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664555f9-b199-4bc2-9eb6-37974ae12c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.432732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664555f9-b199-4bc2-9eb6-37974ae12c27 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.433699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664555f9-b199-4bc2-9eb6-37974ae12c27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:51.435324 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664555f9-b199-4bc2-9eb6-37974ae12c27 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:51.449659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 386/777] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 12 13:14:51 2022] GET /identity/v3/services => generated 772 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:51.450410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3b854-85ed-4b6a-87c5-03c8dbbc35a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:51.451074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d45d41-9c87-44c2-9ed7-4028ecfbfe17 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.451182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3b854-85ed-4b6a-87c5-03c8dbbc35a8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.451363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d45d41-9c87-44c2-9ed7-4028ecfbfe17 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.451462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3b854-85ed-4b6a-87c5-03c8dbbc35a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.451637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a3b854-85ed-4b6a-87c5-03c8dbbc35a8 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.451733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d45d41-9c87-44c2-9ed7-4028ecfbfe17 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.452468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3b854-85ed-4b6a-87c5-03c8dbbc35a8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:51.452777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3b854-85ed-4b6a-87c5-03c8dbbc35a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:51.453341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 387/778] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:51.454220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a3b854-85ed-4b6a-87c5-03c8dbbc35a8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:51.459018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b989036-d2e7-4550-b373-dcdeba62b9fa None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.459556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b989036-d2e7-4550-b373-dcdeba62b9fa None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.459960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b989036-d2e7-4550-b373-dcdeba62b9fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.460848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 392/779] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:14:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:51.466220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3b854-85ed-4b6a-87c5-03c8dbbc35a8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:51.482088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3b989036-d2e7-4550-b373-dcdeba62b9fa None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:51.484412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3b989036-d2e7-4550-b373-dcdeba62b9fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:51.568472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 388/780] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:14:51.580377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a3b854-85ed-4b6a-87c5-03c8dbbc35a8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:51.612950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:51.617923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce844-9297-4c09-9e08-1ceb8f2ae3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:51.618682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce844-9297-4c09-9e08-1ceb8f2ae3dd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.618966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce844-9297-4c09-9e08-1ceb8f2ae3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.619241 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ce844-9297-4c09-9e08-1ceb8f2ae3dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.619786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b989036-d2e7-4550-b373-dcdeba62b9fa None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:51.620170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ce844-9297-4c09-9e08-1ceb8f2ae3dd None admin] RBAC: Adding query filter params (interface=public, service_id=f0de8d29ef5d495a85a990731b50959e, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:51.620506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ce844-9297-4c09-9e08-1ceb8f2ae3dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:51.621933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7ce844-9297-4c09-9e08-1ceb8f2ae3dd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:51.632446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 393/781] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 12 13:14:51 2022] GET /identity/v3/endpoints?service_id=f0de8d29ef5d495a85a990731b50959e&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:51.638288 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce844-9297-4c09-9e08-1ceb8f2ae3dd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:51.752494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b989036-d2e7-4550-b373-dcdeba62b9fa None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:51.757724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ce844-9297-4c09-9e08-1ceb8f2ae3dd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:51.765211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:51.795986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab3136-0082-49f6-9c82-7f995ae80d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:51.797166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab3136-0082-49f6-9c82-7f995ae80d05 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.797633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab3136-0082-49f6-9c82-7f995ae80d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.797972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab3136-0082-49f6-9c82-7f995ae80d05 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.798770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab3136-0082-49f6-9c82-7f995ae80d05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:51.800206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab3136-0082-49f6-9c82-7f995ae80d05 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:51.806032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 394/782] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 12 13:14:51 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:51.810076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48892746-29a5-4cfe-b138-a31fdd68be81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:51.810894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48892746-29a5-4cfe-b138-a31fdd68be81 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:51.812297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48892746-29a5-4cfe-b138-a31fdd68be81 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:51.812417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48892746-29a5-4cfe-b138-a31fdd68be81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:51.834690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48892746-29a5-4cfe-b138-a31fdd68be81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-48892746-29a5-4cfe-b138-a31fdd68be81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:51.843417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:51.844393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:51.845235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 389/783] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:51.851286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab3136-0082-49f6-9c82-7f995ae80d05 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:51.941467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:52.005177 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab3136-0082-49f6-9c82-7f995ae80d05 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:52.056551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9960fc-7c12-4f44-abee-84f97bcba5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:52.057653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9960fc-7c12-4f44-abee-84f97bcba5ff None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:52.058140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9960fc-7c12-4f44-abee-84f97bcba5ff None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:52.058618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9960fc-7c12-4f44-abee-84f97bcba5ff None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:52.059556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9960fc-7c12-4f44-abee-84f97bcba5ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:52.060073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9960fc-7c12-4f44-abee-84f97bcba5ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:52.061987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9960fc-7c12-4f44-abee-84f97bcba5ff None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:52.071067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 395/784] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:52.076376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48892746-29a5-4cfe-b138-a31fdd68be81 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:52.182481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48892746-29a5-4cfe-b138-a31fdd68be81 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:52.224050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5559e37-28a3-45b0-8d61-048a7b438d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:52.225229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5559e37-28a3-45b0-8d61-048a7b438d1b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:52.225736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5559e37-28a3-45b0-8d61-048a7b438d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:52.226215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5559e37-28a3-45b0-8d61-048a7b438d1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:52.245799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5559e37-28a3-45b0-8d61-048a7b438d1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:52.248078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5559e37-28a3-45b0-8d61-048a7b438d1b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d5559e37-28a3-45b0-8d61-048a7b438d1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:52.272885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:52.273844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:52.274764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:52.274764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:52.274764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:52.274764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:52.274764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 390/785] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:52.279987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9960fc-7c12-4f44-abee-84f97bcba5ff None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:52.402596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9960fc-7c12-4f44-abee-84f97bcba5ff None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:52.435040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:52.452347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68003a4-9143-46b7-9873-e5279471a14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:52.453018 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68003a4-9143-46b7-9873-e5279471a14f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:52.453299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68003a4-9143-46b7-9873-e5279471a14f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:52.453603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68003a4-9143-46b7-9873-e5279471a14f None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:52.454334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68003a4-9143-46b7-9873-e5279471a14f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:52.454642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68003a4-9143-46b7-9873-e5279471a14f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:52.456020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68003a4-9143-46b7-9873-e5279471a14f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:52.473115 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 396/786] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:52.478459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5559e37-28a3-45b0-8d61-048a7b438d1b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:52.505476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:52.605263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5559e37-28a3-45b0-8d61-048a7b438d1b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:52.644853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc43f2e-ddfa-419e-9301-a16df7370269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:52.645688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc43f2e-ddfa-419e-9301-a16df7370269 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:52.645979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc43f2e-ddfa-419e-9301-a16df7370269 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:52.646258 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc43f2e-ddfa-419e-9301-a16df7370269 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:52.654950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc43f2e-ddfa-419e-9301-a16df7370269 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:52.657302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc43f2e-ddfa-419e-9301-a16df7370269 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-8fc43f2e-ddfa-419e-9301-a16df7370269 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:52.676969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:52.678014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:52.678958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 391/787] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:52.684622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68003a4-9143-46b7-9873-e5279471a14f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:52.845483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68003a4-9143-46b7-9873-e5279471a14f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:52.910917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291c8ad-134f-4560-bf9d-8fc33947efb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:52.911615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291c8ad-134f-4560-bf9d-8fc33947efb4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:52.911898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291c8ad-134f-4560-bf9d-8fc33947efb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:52.912173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291c8ad-134f-4560-bf9d-8fc33947efb4 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:52.913037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291c8ad-134f-4560-bf9d-8fc33947efb4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:52.913428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291c8ad-134f-4560-bf9d-8fc33947efb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:52.915001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291c8ad-134f-4560-bf9d-8fc33947efb4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:52.940963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 397/788] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 12 13:14:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:52.946296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc43f2e-ddfa-419e-9301-a16df7370269 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:53.047124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9379b5-9356-49fe-a863-ec1785959524 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:53.047471 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9379b5-9356-49fe-a863-ec1785959524 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:53.047674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9379b5-9356-49fe-a863-ec1785959524 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:53.049314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 398/789] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:14:53.058702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad3d27-74ea-4f56-9673-e9cff4b2272d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:53.058988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad3d27-74ea-4f56-9673-e9cff4b2272d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:53.059269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efad3d27-74ea-4f56-9673-e9cff4b2272d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:53.076586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:53.078249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-efad3d27-74ea-4f56-9673-e9cff4b2272d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:53.080507 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-efad3d27-74ea-4f56-9673-e9cff4b2272d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:53.123220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc43f2e-ddfa-419e-9301-a16df7370269 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:53.149722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:53.178720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae4491-f8f4-4ed3-8a7b-bc251945cdba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:53.179929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae4491-f8f4-4ed3-8a7b-bc251945cdba None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:53.180509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae4491-f8f4-4ed3-8a7b-bc251945cdba None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:53.180982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae4491-f8f4-4ed3-8a7b-bc251945cdba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:53.204390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 399/790] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:14:53.207157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae4491-f8f4-4ed3-8a7b-bc251945cdba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:53.208821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae4491-f8f4-4ed3-8a7b-bc251945cdba None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-58ae4491-f8f4-4ed3-8a7b-bc251945cdba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:53.226942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:53.227828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:53.228704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:53.228704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:53.228704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:53.228704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:53.230904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 392/791] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:53.241428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad3d27-74ea-4f56-9673-e9cff4b2272d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:53.244020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae4491-f8f4-4ed3-8a7b-bc251945cdba None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:53.408282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae4491-f8f4-4ed3-8a7b-bc251945cdba None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:53.420624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efad3d27-74ea-4f56-9673-e9cff4b2272d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:53.441989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:53.449102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b1bad-2aed-4232-acdb-b59bbd28fd88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:53.449947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b1bad-2aed-4232-acdb-b59bbd28fd88 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:53.450234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b1bad-2aed-4232-acdb-b59bbd28fd88 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:53.450511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34b1bad-2aed-4232-acdb-b59bbd28fd88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:53.475999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34b1bad-2aed-4232-acdb-b59bbd28fd88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-c34b1bad-2aed-4232-acdb-b59bbd28fd88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:53.484846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:53.485790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:53.486620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:53.490938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 393/792] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:53.492161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b1bad-2aed-4232-acdb-b59bbd28fd88 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:53.522739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8cd58-106d-4a2d-b6c9-377a5d789924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:53.523418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8cd58-106d-4a2d-b6c9-377a5d789924 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:53.523706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8cd58-106d-4a2d-b6c9-377a5d789924 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:53.523986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e8cd58-106d-4a2d-b6c9-377a5d789924 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:53.524734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8cd58-106d-4a2d-b6c9-377a5d789924 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:53.525043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8cd58-106d-4a2d-b6c9-377a5d789924 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:53.526458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e8cd58-106d-4a2d-b6c9-377a5d789924 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:53.547775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 400/793] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:53.555241 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8cd58-106d-4a2d-b6c9-377a5d789924 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:53.618407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34b1bad-2aed-4232-acdb-b59bbd28fd88 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:53.620011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:53.653329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47d499-6d94-4e1d-92ca-af9d028e2f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:53.654027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47d499-6d94-4e1d-92ca-af9d028e2f10 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:53.654316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47d499-6d94-4e1d-92ca-af9d028e2f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:53.654596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac47d499-6d94-4e1d-92ca-af9d028e2f10 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:53.655399 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47d499-6d94-4e1d-92ca-af9d028e2f10 None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:53.655792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47d499-6d94-4e1d-92ca-af9d028e2f10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:53.657175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac47d499-6d94-4e1d-92ca-af9d028e2f10 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:53.669564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 394/794] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:53.671784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47d499-6d94-4e1d-92ca-af9d028e2f10 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:53.737679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:53.743240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e8cd58-106d-4a2d-b6c9-377a5d789924 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:53.780527 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac47d499-6d94-4e1d-92ca-af9d028e2f10 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:53.819135 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c66e32-b417-4a0c-9813-b50fa303ee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:53.819745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fa606-7976-4236-b2b4-4fb1e1e632d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:53.820545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fa606-7976-4236-b2b4-4fb1e1e632d1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:53.821050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c66e32-b417-4a0c-9813-b50fa303ee74 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:53.821336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c66e32-b417-4a0c-9813-b50fa303ee74 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:53.821628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c66e32-b417-4a0c-9813-b50fa303ee74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:53.821950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fa606-7976-4236-b2b4-4fb1e1e632d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:53.822232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327fa606-7976-4236-b2b4-4fb1e1e632d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:53.834793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fa606-7976-4236-b2b4-4fb1e1e632d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:53.835278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c66e32-b417-4a0c-9813-b50fa303ee74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:53.836731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327fa606-7976-4236-b2b4-4fb1e1e632d1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:53.836840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c66e32-b417-4a0c-9813-b50fa303ee74 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-327fa606-7976-4236-b2b4-4fb1e1e632d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:53.851615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:53.852517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:53.853785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:53.853785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:53.853785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:53.853785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:53.853785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:53.853785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:53.853785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:53.853785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-86c66e32-b417-4a0c-9813-b50fa303ee74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:53.854664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:53.855478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:53.856338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:53.856338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:53.856338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:53.856338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:53.861551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 395/796] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:53.861551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf15b6a-25b4-463b-b40e-6de2d15cf9f9 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:53.861551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf15b6a-25b4-463b-b40e-6de2d15cf9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:53.861950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 401/796] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:53.864117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf15b6a-25b4-463b-b40e-6de2d15cf9f9 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:53.865073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 396/797] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:53.865621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fa606-7976-4236-b2b4-4fb1e1e632d1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:53.865798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf15b6a-25b4-463b-b40e-6de2d15cf9f9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:53.966417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:53.977868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327fa606-7976-4236-b2b4-4fb1e1e632d1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:53.979369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf15b6a-25b4-463b-b40e-6de2d15cf9f9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:53.986821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:54.017994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850011b-51ee-4713-a3e9-ec0bbb86058d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:54.018202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10b71c-749e-41df-9151-2cf8ff9e61b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:54.018815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10b71c-749e-41df-9151-2cf8ff9e61b2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.019102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10b71c-749e-41df-9151-2cf8ff9e61b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.019383 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10b71c-749e-41df-9151-2cf8ff9e61b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.019775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850011b-51ee-4713-a3e9-ec0bbb86058d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.020113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10b71c-749e-41df-9151-2cf8ff9e61b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:54.020422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10b71c-749e-41df-9151-2cf8ff9e61b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:54.020664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850011b-51ee-4713-a3e9-ec0bbb86058d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.021124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4850011b-51ee-4713-a3e9-ec0bbb86058d None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.021817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10b71c-749e-41df-9151-2cf8ff9e61b2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:54.022336 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850011b-51ee-4713-a3e9-ec0bbb86058d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:54.022854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850011b-51ee-4713-a3e9-ec0bbb86058d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:54.024722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4850011b-51ee-4713-a3e9-ec0bbb86058d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:54.041440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 397/798] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:54.041440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b812ba0-8d07-4dde-9358-966bbc6f6357 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.041440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b812ba0-8d07-4dde-9358-966bbc6f6357 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.041892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b812ba0-8d07-4dde-9358-966bbc6f6357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.042310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 402/799] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:54.044350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850011b-51ee-4713-a3e9-ec0bbb86058d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:54.060966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-2b812ba0-8d07-4dde-9358-966bbc6f6357 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:54.063127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2b812ba0-8d07-4dde-9358-966bbc6f6357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:54.138632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 398/800] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:14:54.140042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b812ba0-8d07-4dde-9358-966bbc6f6357 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:54.153472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4850011b-51ee-4713-a3e9-ec0bbb86058d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:54.190010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62db6eba-ecc0-4786-82ef-5346bee06857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:54.190903 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db6eba-ecc0-4786-82ef-5346bee06857 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.191186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db6eba-ecc0-4786-82ef-5346bee06857 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.191458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62db6eba-ecc0-4786-82ef-5346bee06857 None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b69264fe173045e08c282171cd7370af/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.193466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db6eba-ecc0-4786-82ef-5346bee06857 None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=b69264fe173045e08c282171cd7370af, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:54.194909 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62db6eba-ecc0-4786-82ef-5346bee06857 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:54.208573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 403/801] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:14:54 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b69264fe173045e08c282171cd7370af/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:14:54.219700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86297c6-f155-4348-b130-cf7e3bfdbda7 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.220343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86297c6-f155-4348-b130-cf7e3bfdbda7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.221144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86297c6-f155-4348-b130-cf7e3bfdbda7 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.224459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 404/802] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:54.225763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86297c6-f155-4348-b130-cf7e3bfdbda7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:54.248116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:54.264030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:54.272541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b812ba0-8d07-4dde-9358-966bbc6f6357 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:54.325022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64616de9-075a-4853-b4b3-48d6ea59a8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:54.326530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64616de9-075a-4853-b4b3-48d6ea59a8bb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.327100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64616de9-075a-4853-b4b3-48d6ea59a8bb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.327709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64616de9-075a-4853-b4b3-48d6ea59a8bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.339408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64616de9-075a-4853-b4b3-48d6ea59a8bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:54.341984 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64616de9-075a-4853-b4b3-48d6ea59a8bb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-64616de9-075a-4853-b4b3-48d6ea59a8bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:54.365070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:54.366360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:14:54.367520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:54.373891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 399/803] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:14:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:54.375021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69311896-9570-490c-866d-ab089b8cd53e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.375313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69311896-9570-490c-866d-ab089b8cd53e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.375594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69311896-9570-490c-866d-ab089b8cd53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.377575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86297c6-f155-4348-b130-cf7e3bfdbda7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:54.394707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-69311896-9570-490c-866d-ab089b8cd53e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:54.395912 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-69311896-9570-490c-866d-ab089b8cd53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:54.419685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de24bd-6e5c-421f-8aec-c3dc4500ff90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:54.420596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de24bd-6e5c-421f-8aec-c3dc4500ff90 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.421941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de24bd-6e5c-421f-8aec-c3dc4500ff90 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.422506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de24bd-6e5c-421f-8aec-c3dc4500ff90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.438831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3de24bd-6e5c-421f-8aec-c3dc4500ff90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d3de24bd-6e5c-421f-8aec-c3dc4500ff90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:54.448740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:54.449734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:54.450560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:54.451206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 405/804] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:54.456949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de24bd-6e5c-421f-8aec-c3dc4500ff90 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:54.480774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 400/805] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:14:54.482138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69311896-9570-490c-866d-ab089b8cd53e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:54.563860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:54.579809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:54.597243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de24bd-6e5c-421f-8aec-c3dc4500ff90 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:54.597950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69311896-9570-490c-866d-ab089b8cd53e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:54.635892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32de6548-82bb-45f7-b20d-74e61afc2858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:54.636569 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de6548-82bb-45f7-b20d-74e61afc2858 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.636858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de6548-82bb-45f7-b20d-74e61afc2858 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.636978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e9255-7b5d-4504-8516-7b0e9ec52176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:54.637150 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32de6548-82bb-45f7-b20d-74e61afc2858 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.637606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e9255-7b5d-4504-8516-7b0e9ec52176 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.637892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e9255-7b5d-4504-8516-7b0e9ec52176 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.638045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de6548-82bb-45f7-b20d-74e61afc2858 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:54.638155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4e9255-7b5d-4504-8516-7b0e9ec52176 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.638325 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de6548-82bb-45f7-b20d-74e61afc2858 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:54.638908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e9255-7b5d-4504-8516-7b0e9ec52176 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:54.639218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e9255-7b5d-4504-8516-7b0e9ec52176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:54.639948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32de6548-82bb-45f7-b20d-74e61afc2858 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:54.640616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4e9255-7b5d-4504-8516-7b0e9ec52176 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:54.649891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 401/806] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:54.654855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e9255-7b5d-4504-8516-7b0e9ec52176 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:54.659447 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 406/807] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 12 13:14:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:54.660766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfd70c-cef1-454d-801a-2e30405f4a5b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.661067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfd70c-cef1-454d-801a-2e30405f4a5b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.661346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfd70c-cef1-454d-801a-2e30405f4a5b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.662712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 407/808] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:54.667180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfd70c-cef1-454d-801a-2e30405f4a5b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:54.771437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4e9255-7b5d-4504-8516-7b0e9ec52176 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:54.773126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bfd70c-cef1-454d-801a-2e30405f4a5b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:54.805638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:54.809296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d27ba-8dc9-4dca-bbee-6093565438fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:54.810322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d27ba-8dc9-4dca-bbee-6093565438fd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.810691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d27ba-8dc9-4dca-bbee-6093565438fd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.810961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9d27ba-8dc9-4dca-bbee-6093565438fd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.811969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d27ba-8dc9-4dca-bbee-6093565438fd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:54.813334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9d27ba-8dc9-4dca-bbee-6093565438fd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:54.815688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb19a35-77b3-4396-9761-33c388fe12ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:54.816704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb19a35-77b3-4396-9761-33c388fe12ac None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.817200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb19a35-77b3-4396-9761-33c388fe12ac None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.817696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb19a35-77b3-4396-9761-33c388fe12ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7a9d27ba-8dc9-4dca-bbee-6093565438fd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:54.826138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:54.827425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:54.828252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 402/809] 10.222.0.23 () {56 vars in 1129 bytes} [Tue Jul 12 13:14:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:54.829099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d27ba-8dc9-4dca-bbee-6093565438fd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:54.835159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb19a35-77b3-4396-9761-33c388fe12ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:54.836820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb19a35-77b3-4396-9761-33c388fe12ac None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:54.847941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2fb19a35-77b3-4396-9761-33c388fe12ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:54.856360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:54.857214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:54.858192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:54.858192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:54.858192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:54.858192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:54.860639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 408/810] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:54.861863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062ee14-cb06-4dd1-a6e5-3c83ed1b3885 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:54.862151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062ee14-cb06-4dd1-a6e5-3c83ed1b3885 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:54.862424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062ee14-cb06-4dd1-a6e5-3c83ed1b3885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:54.879832 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e062ee14-cb06-4dd1-a6e5-3c83ed1b3885 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:54.881031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e062ee14-cb06-4dd1-a6e5-3c83ed1b3885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:54.957425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 409/811] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:14:54.957425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062ee14-cb06-4dd1-a6e5-3c83ed1b3885 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:54.963484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9d27ba-8dc9-4dca-bbee-6093565438fd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.005954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae96e39-cdbd-4725-9aa3-81f863bd3209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.007171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae96e39-cdbd-4725-9aa3-81f863bd3209 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.007460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae96e39-cdbd-4725-9aa3-81f863bd3209 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.007738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae96e39-cdbd-4725-9aa3-81f863bd3209 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.022109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae96e39-cdbd-4725-9aa3-81f863bd3209 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.023737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae96e39-cdbd-4725-9aa3-81f863bd3209 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-eae96e39-cdbd-4725-9aa3-81f863bd3209 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:55.042082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:55.042977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:55.043861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:55.043861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:55.043861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:55.043861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:55.043861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 403/812] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:55.044141 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae96e39-cdbd-4725-9aa3-81f863bd3209 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.073750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062ee14-cb06-4dd1-a6e5-3c83ed1b3885 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.087893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:55.108042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:55.115848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb0976-96a0-4ec0-92bc-a83edec92b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.116515 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb0976-96a0-4ec0-92bc-a83edec92b92 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.116805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb0976-96a0-4ec0-92bc-a83edec92b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.117083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb0976-96a0-4ec0-92bc-a83edec92b92 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.117842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb0976-96a0-4ec0-92bc-a83edec92b92 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:55.118148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb0976-96a0-4ec0-92bc-a83edec92b92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.119689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddb0976-96a0-4ec0-92bc-a83edec92b92 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.124913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 410/813] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 12 13:14:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:55.125821 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb0976-96a0-4ec0-92bc-a83edec92b92 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.154246 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae96e39-cdbd-4725-9aa3-81f863bd3209 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.195602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9a27-6a8b-438a-b882-6c35a924dbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.197203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9a27-6a8b-438a-b882-6c35a924dbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.197700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9a27-6a8b-438a-b882-6c35a924dbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.198156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69f9a27-6a8b-438a-b882-6c35a924dbf1 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.199075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f9a27-6a8b-438a-b882-6c35a924dbf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:55.199570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f9a27-6a8b-438a-b882-6c35a924dbf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.201522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69f9a27-6a8b-438a-b882-6c35a924dbf1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.220921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 404/814] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:55.220921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9a27-6a8b-438a-b882-6c35a924dbf1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.233496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddb0976-96a0-4ec0-92bc-a83edec92b92 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.272097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661547-c02f-40da-b808-a9dcfdb0ef43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.272952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661547-c02f-40da-b808-a9dcfdb0ef43 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.273276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661547-c02f-40da-b808-a9dcfdb0ef43 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.273473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87661547-c02f-40da-b808-a9dcfdb0ef43 None admin] PATH_INFO: `/v3/services/network` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.274132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87661547-c02f-40da-b808-a9dcfdb0ef43 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.275492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87661547-c02f-40da-b808-a9dcfdb0ef43 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-87661547-c02f-40da-b808-a9dcfdb0ef43 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:55.287740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:55.288723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:55.289844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 411/815] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/services/network => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:55.290644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661547-c02f-40da-b808-a9dcfdb0ef43 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.331497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:55.338266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69f9a27-6a8b-438a-b882-6c35a924dbf1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.363929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:55.376206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3119954-b5a0-4f18-8ea0-66f2b3125082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.376892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3119954-b5a0-4f18-8ea0-66f2b3125082 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.377180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3119954-b5a0-4f18-8ea0-66f2b3125082 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.377488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3119954-b5a0-4f18-8ea0-66f2b3125082 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.378252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3119954-b5a0-4f18-8ea0-66f2b3125082 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:55.378566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3119954-b5a0-4f18-8ea0-66f2b3125082 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.380000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3119954-b5a0-4f18-8ea0-66f2b3125082 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.393864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 405/816] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:55.394716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3119954-b5a0-4f18-8ea0-66f2b3125082 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.405085 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87661547-c02f-40da-b808-a9dcfdb0ef43 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.444273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d0c77-93c6-40e2-9014-955e63635ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.445682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d0c77-93c6-40e2-9014-955e63635ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.445870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d0c77-93c6-40e2-9014-955e63635ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.446145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d0c77-93c6-40e2-9014-955e63635ae1 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.446867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d0c77-93c6-40e2-9014-955e63635ae1 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:55.447175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d0c77-93c6-40e2-9014-955e63635ae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.448543 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d0c77-93c6-40e2-9014-955e63635ae1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.454325 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 412/817] 10.222.0.23 () {56 vars in 1124 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:55.455109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d0c77-93c6-40e2-9014-955e63635ae1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.512724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3119954-b5a0-4f18-8ea0-66f2b3125082 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.549615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3119d70-497a-4c92-be21-0ed95376c2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.550347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3119d70-497a-4c92-be21-0ed95376c2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.550629 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3119d70-497a-4c92-be21-0ed95376c2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.550901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3119d70-497a-4c92-be21-0ed95376c2c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.558650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3119d70-497a-4c92-be21-0ed95376c2c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.559755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d0c77-93c6-40e2-9014-955e63635ae1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.560087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3119d70-497a-4c92-be21-0ed95376c2c8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.563004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f3119d70-497a-4c92-be21-0ed95376c2c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:55.575940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:55.576882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:55.578464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 406/818] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:55.579049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3119d70-497a-4c92-be21-0ed95376c2c8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.604730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfd12e-de5e-446f-bc83-66ea7681143b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.605418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfd12e-de5e-446f-bc83-66ea7681143b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.605712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfd12e-de5e-446f-bc83-66ea7681143b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.606143 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfd12e-de5e-446f-bc83-66ea7681143b None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.606701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfd12e-de5e-446f-bc83-66ea7681143b None admin] RBAC: Adding query filter params (name=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:55.607016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfd12e-de5e-446f-bc83-66ea7681143b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.608481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfd12e-de5e-446f-bc83-66ea7681143b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.617761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 413/819] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:55.618454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfd12e-de5e-446f-bc83-66ea7681143b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.629282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:55.685455 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3119d70-497a-4c92-be21-0ed95376c2c8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.723608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3909dc-ecd5-489c-92a0-bde24afd828e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.724436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3909dc-ecd5-489c-92a0-bde24afd828e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.724725 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3909dc-ecd5-489c-92a0-bde24afd828e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.725005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3909dc-ecd5-489c-92a0-bde24afd828e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.725962 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfd12e-de5e-446f-bc83-66ea7681143b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.731946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3909dc-ecd5-489c-92a0-bde24afd828e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.733699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3909dc-ecd5-489c-92a0-bde24afd828e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-9d3909dc-ecd5-489c-92a0-bde24afd828e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:55.748351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:55.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:55.750742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:55.750742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:55.750742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:55.750742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:55.750742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:55.750742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:55.750742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:55.750742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:55.750742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 407/820] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:55.751833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3909dc-ecd5-489c-92a0-bde24afd828e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.768923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b15206-9a04-4ce4-a734-4225b711e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.769627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b15206-9a04-4ce4-a734-4225b711e9c7 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.769918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b15206-9a04-4ce4-a734-4225b711e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.770193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b15206-9a04-4ce4-a734-4225b711e9c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.770932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b15206-9a04-4ce4-a734-4225b711e9c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.772295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b15206-9a04-4ce4-a734-4225b711e9c7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.793113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 414/821] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 12 13:14:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:55.793965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b15206-9a04-4ce4-a734-4225b711e9c7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.837071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:55.886094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3909dc-ecd5-489c-92a0-bde24afd828e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:55.891255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:55.939165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507a228-3722-48c1-88f6-f4e5c7fcb8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:55.939880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507a228-3722-48c1-88f6-f4e5c7fcb8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:55.940212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507a228-3722-48c1-88f6-f4e5c7fcb8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:55.940424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507a228-3722-48c1-88f6-f4e5c7fcb8f8 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:55.941284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507a228-3722-48c1-88f6-f4e5c7fcb8f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:55.941674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507a228-3722-48c1-88f6-f4e5c7fcb8f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:55.943148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507a228-3722-48c1-88f6-f4e5c7fcb8f8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:55.966506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 408/822] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:55.966506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507a228-3722-48c1-88f6-f4e5c7fcb8f8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:55.966803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b15206-9a04-4ce4-a734-4225b711e9c7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:56.010262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d785224-51e6-49ec-8c7f-7d36c1d177ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:56.011281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d785224-51e6-49ec-8c7f-7d36c1d177ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.011781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d785224-51e6-49ec-8c7f-7d36c1d177ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.013577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d785224-51e6-49ec-8c7f-7d36c1d177ec None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.013577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d785224-51e6-49ec-8c7f-7d36c1d177ec None admin] RBAC: Adding query filter params (type=network) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:56.013874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d785224-51e6-49ec-8c7f-7d36c1d177ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:56.015910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d785224-51e6-49ec-8c7f-7d36c1d177ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:56.022501 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 415/823] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/services?type=network => generated 350 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:56.023953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d785224-51e6-49ec-8c7f-7d36c1d177ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:56.104338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507a228-3722-48c1-88f6-f4e5c7fcb8f8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:56.138077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:56.146222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f9bcb-4438-4560-b3fa-78978609ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:56.147020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f9bcb-4438-4560-b3fa-78978609ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.147491 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f9bcb-4438-4560-b3fa-78978609ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.147869 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f9bcb-4438-4560-b3fa-78978609ecf8 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.148868 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f9bcb-4438-4560-b3fa-78978609ecf8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:56.149279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f9bcb-4438-4560-b3fa-78978609ecf8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:56.151003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f9bcb-4438-4560-b3fa-78978609ecf8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:56.159800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d785224-51e6-49ec-8c7f-7d36c1d177ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:56.165080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 409/824] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:56.166059 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f9bcb-4438-4560-b3fa-78978609ecf8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:56.194367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:56.204305 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1149f7-6278-496b-bdc3-322be6fe3eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:56.205101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1149f7-6278-496b-bdc3-322be6fe3eab None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.205510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1149f7-6278-496b-bdc3-322be6fe3eab None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.205890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1149f7-6278-496b-bdc3-322be6fe3eab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.287504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1149f7-6278-496b-bdc3-322be6fe3eab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:56.289173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1149f7-6278-496b-bdc3-322be6fe3eab None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:56.301913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f9bcb-4438-4560-b3fa-78978609ecf8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:56.344938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f384bdc-3e90-45c6-8b92-eb25c1dcf727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:56.345636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f384bdc-3e90-45c6-8b92-eb25c1dcf727 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.345929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f384bdc-3e90-45c6-8b92-eb25c1dcf727 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.346212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f384bdc-3e90-45c6-8b92-eb25c1dcf727 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.346959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f384bdc-3e90-45c6-8b92-eb25c1dcf727 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:56.348341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f384bdc-3e90-45c6-8b92-eb25c1dcf727 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:56.379537 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 410/825] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 12 13:14:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:14:56.379537 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f384bdc-3e90-45c6-8b92-eb25c1dcf727 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:56.382197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:56.396861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 416/826] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:56 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:14:56.398335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52ef04-d752-4ba5-9059-61cb2f9ef403 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.398725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52ef04-d752-4ba5-9059-61cb2f9ef403 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.399098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b52ef04-d752-4ba5-9059-61cb2f9ef403 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.404504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 417/827] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:56.405453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52ef04-d752-4ba5-9059-61cb2f9ef403 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:56.487977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:56.537408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f384bdc-3e90-45c6-8b92-eb25c1dcf727 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:56.550881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b52ef04-d752-4ba5-9059-61cb2f9ef403 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:56.585341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca643f-6d74-494c-acd0-b8706d585b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:56.586306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca643f-6d74-494c-acd0-b8706d585b6f None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.586598 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca643f-6d74-494c-acd0-b8706d585b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.586875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca643f-6d74-494c-acd0-b8706d585b6f None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b26fa3bc9c0d4e22a8abc5a4d98c9995/roles/bd68bb224f0b4111bf407cf17e7143a4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.596332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88299f3-2cc3-49f1-8ba3-a50874df9e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:56.597354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88299f3-2cc3-49f1-8ba3-a50874df9e70 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.597722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88299f3-2cc3-49f1-8ba3-a50874df9e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.598392 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88299f3-2cc3-49f1-8ba3-a50874df9e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.603081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca643f-6d74-494c-acd0-b8706d585b6f None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=b26fa3bc9c0d4e22a8abc5a4d98c9995, role_id=bd68bb224f0b4111bf407cf17e7143a4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:56.604536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca643f-6d74-494c-acd0-b8706d585b6f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:56.612486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88299f3-2cc3-49f1-8ba3-a50874df9e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:56.614142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88299f3-2cc3-49f1-8ba3-a50874df9e70 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:56.625779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 411/828] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:14:56 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b26fa3bc9c0d4e22a8abc5a4d98c9995/roles/bd68bb224f0b4111bf407cf17e7143a4 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:14:56.627634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dc71e-f70c-4a9e-a8a0-9d1feca239fb None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.628067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dc71e-f70c-4a9e-a8a0-9d1feca239fb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.628488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76dc71e-f70c-4a9e-a8a0-9d1feca239fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b88299f3-2cc3-49f1-8ba3-a50874df9e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:56.638675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:56.640357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:56.642221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:56.642221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:56.642221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:56.642221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:56.642221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 418/829] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:56.646377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88299f3-2cc3-49f1-8ba3-a50874df9e70 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:56.649448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a76dc71e-f70c-4a9e-a8a0-9d1feca239fb None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:56.650980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a76dc71e-f70c-4a9e-a8a0-9d1feca239fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:56.690164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:56.762814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 412/830] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:14:56.778633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88299f3-2cc3-49f1-8ba3-a50874df9e70 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:56.786973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76dc71e-f70c-4a9e-a8a0-9d1feca239fb None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:56.818006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7223f-6e9f-4dea-ada4-0de068f738da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:56.819020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7223f-6e9f-4dea-ada4-0de068f738da None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.819504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7223f-6e9f-4dea-ada4-0de068f738da None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.820003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd7223f-6e9f-4dea-ada4-0de068f738da None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.820925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7223f-6e9f-4dea-ada4-0de068f738da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:56.821481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7223f-6e9f-4dea-ada4-0de068f738da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:56.823140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd7223f-6e9f-4dea-ada4-0de068f738da None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:56.839531 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 419/831] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:56.845183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7223f-6e9f-4dea-ada4-0de068f738da None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:56.850391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:56.929140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76dc71e-f70c-4a9e-a8a0-9d1feca239fb None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:56.954652 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:56.962036 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd7223f-6e9f-4dea-ada4-0de068f738da None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:56.975430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602520e7-d10e-4a89-9059-77b37f0d14f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:56.976326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602520e7-d10e-4a89-9059-77b37f0d14f0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:56.976681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602520e7-d10e-4a89-9059-77b37f0d14f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:56.977028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602520e7-d10e-4a89-9059-77b37f0d14f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:56.988179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602520e7-d10e-4a89-9059-77b37f0d14f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-602520e7-d10e-4a89-9059-77b37f0d14f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:56.996274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:56.997538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:14:56.998588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:57.000751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 413/832] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:57.006092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602520e7-d10e-4a89-9059-77b37f0d14f0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:57.010539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b55fa-3cd9-40a5-b3d5-f9550149cc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:57.011548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b55fa-3cd9-40a5-b3d5-f9550149cc69 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:57.011987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b55fa-3cd9-40a5-b3d5-f9550149cc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:57.012418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b55fa-3cd9-40a5-b3d5-f9550149cc69 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:57.022757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b55fa-3cd9-40a5-b3d5-f9550149cc69 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:57.025282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21b55fa-3cd9-40a5-b3d5-f9550149cc69 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d21b55fa-3cd9-40a5-b3d5-f9550149cc69 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:57.048067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:57.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:57.050277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:57.050277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:57.050277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:57.050277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:57.050277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:57.050277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:57.050277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:57.050277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:57.052912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 420/833] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:57.062784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b55fa-3cd9-40a5-b3d5-f9550149cc69 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:57.121562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602520e7-d10e-4a89-9059-77b37f0d14f0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:57.133622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:57.162541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bffcba-b59c-44c7-976d-3d94fece1695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:57.163330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bffcba-b59c-44c7-976d-3d94fece1695 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:57.163671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bffcba-b59c-44c7-976d-3d94fece1695 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:57.164018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bffcba-b59c-44c7-976d-3d94fece1695 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:57.164854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bffcba-b59c-44c7-976d-3d94fece1695 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:57.165223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bffcba-b59c-44c7-976d-3d94fece1695 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:57.166711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bffcba-b59c-44c7-976d-3d94fece1695 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:57.180944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 414/834] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:14:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:57.181863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bffcba-b59c-44c7-976d-3d94fece1695 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:57.196044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21b55fa-3cd9-40a5-b3d5-f9550149cc69 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:57.247372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77e3c4-a11c-4300-9562-5332000505fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:57.248038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77e3c4-a11c-4300-9562-5332000505fd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:57.248322 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77e3c4-a11c-4300-9562-5332000505fd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:57.248598 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77e3c4-a11c-4300-9562-5332000505fd None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:57.249573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77e3c4-a11c-4300-9562-5332000505fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:57.249887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77e3c4-a11c-4300-9562-5332000505fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:57.251323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77e3c4-a11c-4300-9562-5332000505fd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:57.263989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 421/835] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:57.269766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77e3c4-a11c-4300-9562-5332000505fd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:57.285398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:57.315707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bffcba-b59c-44c7-976d-3d94fece1695 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:57.353317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555dd43-dd03-43a2-9dd2-4d6468644a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:57.354639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555dd43-dd03-43a2-9dd2-4d6468644a92 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:57.355076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555dd43-dd03-43a2-9dd2-4d6468644a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:57.355531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555dd43-dd03-43a2-9dd2-4d6468644a92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:57.374358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77e3c4-a11c-4300-9562-5332000505fd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:57.414273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89284bba-7025-4894-8ee5-e0a3280d7af5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:57.415186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89284bba-7025-4894-8ee5-e0a3280d7af5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:57.415567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89284bba-7025-4894-8ee5-e0a3280d7af5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:57.415940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89284bba-7025-4894-8ee5-e0a3280d7af5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:57.416898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89284bba-7025-4894-8ee5-e0a3280d7af5 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=1cc0050f14664630a36814ca670f124f) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:57.417318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89284bba-7025-4894-8ee5-e0a3280d7af5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:57.418934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89284bba-7025-4894-8ee5-e0a3280d7af5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:57.429424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 422/836] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:57 2022] GET /identity/v3/role_assignments?user.id=1cc0050f14664630a36814ca670f124f&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:57.497696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555dd43-dd03-43a2-9dd2-4d6468644a92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:57.499933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555dd43-dd03-43a2-9dd2-4d6468644a92 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-c555dd43-dd03-43a2-9dd2-4d6468644a92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:57.519942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:57.520893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:57.521808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:57.521808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:57.521808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:57.521808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:57.521808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 415/837] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:57.528222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89284bba-7025-4894-8ee5-e0a3280d7af5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:57.614459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:57.636805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89284bba-7025-4894-8ee5-e0a3280d7af5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:57.672932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0a5e6-f354-489b-894c-82a5a2654e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:57.673779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0a5e6-f354-489b-894c-82a5a2654e73 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:57.674160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0a5e6-f354-489b-894c-82a5a2654e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:57.674533 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0a5e6-f354-489b-894c-82a5a2654e73 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:57.675370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0a5e6-f354-489b-894c-82a5a2654e73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:57.675776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0a5e6-f354-489b-894c-82a5a2654e73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:57.677237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c0a5e6-f354-489b-894c-82a5a2654e73 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:57.693779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 423/838] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:57.695406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555dd43-dd03-43a2-9dd2-4d6468644a92 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:57.722178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:57.800823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555dd43-dd03-43a2-9dd2-4d6468644a92 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:57.839179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ff9d0e-1e10-407f-8351-affc1b77e648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:57.840373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff9d0e-1e10-407f-8351-affc1b77e648 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:57.840862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff9d0e-1e10-407f-8351-affc1b77e648 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:57.841343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ff9d0e-1e10-407f-8351-affc1b77e648 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:57.849821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff9d0e-1e10-407f-8351-affc1b77e648 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:57.851625 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ff9d0e-1e10-407f-8351-affc1b77e648 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-c5ff9d0e-1e10-407f-8351-affc1b77e648 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:57.867590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:57.868547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 12 13:14:57.869501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:57.872099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 416/839] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:14:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:57.877685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0a5e6-f354-489b-894c-82a5a2654e73 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:57.991296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0a5e6-f354-489b-894c-82a5a2654e73 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:58.006526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7324df-94c2-47eb-9c4d-03ab225bd61c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.006829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7324df-94c2-47eb-9c4d-03ab225bd61c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.007079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7324df-94c2-47eb-9c4d-03ab225bd61c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.008626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 417/840] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:14:58.013814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d45cd-359a-4af0-8a12-d9035a056ef6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.014359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d45cd-359a-4af0-8a12-d9035a056ef6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.015569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860d45cd-359a-4af0-8a12-d9035a056ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.032024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cce1e-8ca8-42e8-8921-ffecc2a846a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:58.032700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cce1e-8ca8-42e8-8921-ffecc2a846a4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.032985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cce1e-8ca8-42e8-8921-ffecc2a846a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.033256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cce1e-8ca8-42e8-8921-ffecc2a846a4 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.033714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-860d45cd-359a-4af0-8a12-d9035a056ef6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:58.034141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747cce1e-8ca8-42e8-8921-ffecc2a846a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:58.034451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747cce1e-8ca8-42e8-8921-ffecc2a846a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:58.035572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-860d45cd-359a-4af0-8a12-d9035a056ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:58.035936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747cce1e-8ca8-42e8-8921-ffecc2a846a4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:58.053799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:58.062348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 424/841] 10.222.0.23 () {56 vars in 1135 bytes} [Tue Jul 12 13:14:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:58.064141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cce1e-8ca8-42e8-8921-ffecc2a846a4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:58.126752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 418/842] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:14:58.146296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d45cd-359a-4af0-8a12-d9035a056ef6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:58.164442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:58.197692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cce1e-8ca8-42e8-8921-ffecc2a846a4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:58.240060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d7851-e21c-43a0-9fc1-15aa66f36811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:58.240716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d7851-e21c-43a0-9fc1-15aa66f36811 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.241003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d7851-e21c-43a0-9fc1-15aa66f36811 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.241282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328d7851-e21c-43a0-9fc1-15aa66f36811 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.271266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860d45cd-359a-4af0-8a12-d9035a056ef6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:58.309299 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f9a19-50ec-40e1-9251-3a4add781dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:58.310231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f9a19-50ec-40e1-9251-3a4add781dfc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.310579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f9a19-50ec-40e1-9251-3a4add781dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.310950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06f9a19-50ec-40e1-9251-3a4add781dfc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.312090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f9a19-50ec-40e1-9251-3a4add781dfc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:58.313511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:58.314712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06f9a19-50ec-40e1-9251-3a4add781dfc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:58.320538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d7851-e21c-43a0-9fc1-15aa66f36811 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:58.322073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328d7851-e21c-43a0-9fc1-15aa66f36811 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-a06f9a19-50ec-40e1-9251-3a4add781dfc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:58.329328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:58.330435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:14:58.331259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:58.339146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 419/843] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:58.340966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f9a19-50ec-40e1-9251-3a4add781dfc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:58.437253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:58.463484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06f9a19-50ec-40e1-9251-3a4add781dfc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:58.501883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6e0b-2031-4a18-9792-0721ad64e0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:58.502950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6e0b-2031-4a18-9792-0721ad64e0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.503446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6e0b-2031-4a18-9792-0721ad64e0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.503981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fde6e0b-2031-4a18-9792-0721ad64e0d0 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.505008 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6e0b-2031-4a18-9792-0721ad64e0d0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:58.505567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6e0b-2031-4a18-9792-0721ad64e0d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:58.507153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fde6e0b-2031-4a18-9792-0721ad64e0d0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:58.517505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 420/844] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:58.519102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6e0b-2031-4a18-9792-0721ad64e0d0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:58.550820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 425/845] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:14:58 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:14:58.552763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d7851-e21c-43a0-9fc1-15aa66f36811 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:58.636821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fde6e0b-2031-4a18-9792-0721ad64e0d0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:58.648992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:58.655339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328d7851-e21c-43a0-9fc1-15aa66f36811 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:58.681885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e04e2f-eea2-4033-94e2-e5046fe2079d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:58.682918 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e04e2f-eea2-4033-94e2-e5046fe2079d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.683433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e04e2f-eea2-4033-94e2-e5046fe2079d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.684004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e04e2f-eea2-4033-94e2-e5046fe2079d None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.685030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e04e2f-eea2-4033-94e2-e5046fe2079d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:58.685584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e04e2f-eea2-4033-94e2-e5046fe2079d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:58.688070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e04e2f-eea2-4033-94e2-e5046fe2079d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:58.694820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984fcab-9ff7-4f25-bb1c-49c5fd823889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:58.694820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984fcab-9ff7-4f25-bb1c-49c5fd823889 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.695049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984fcab-9ff7-4f25-bb1c-49c5fd823889 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.701577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984fcab-9ff7-4f25-bb1c-49c5fd823889 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.701796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 421/846] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:14:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:58.714799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984fcab-9ff7-4f25-bb1c-49c5fd823889 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:58.716303 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984fcab-9ff7-4f25-bb1c-49c5fd823889 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e984fcab-9ff7-4f25-bb1c-49c5fd823889 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:58.735843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:58.736811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:58.737919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:58.737919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:58.737919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:58.737919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:58.737919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 426/847] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:58.744498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e04e2f-eea2-4033-94e2-e5046fe2079d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:58.755272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:58.758369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740cd83-6b42-4d7e-9882-b822b6ef105d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.758650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740cd83-6b42-4d7e-9882-b822b6ef105d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.758920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740cd83-6b42-4d7e-9882-b822b6ef105d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.760582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 427/848] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:58.766798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587a241-c1d0-4e2f-9100-ca4fd2b1fa63 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.767089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587a241-c1d0-4e2f-9100-ca4fd2b1fa63 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.767367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5587a241-c1d0-4e2f-9100-ca4fd2b1fa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.785525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5587a241-c1d0-4e2f-9100-ca4fd2b1fa63 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:58.786750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5587a241-c1d0-4e2f-9100-ca4fd2b1fa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:58.869781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 428/849] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:14:58.873610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e04e2f-eea2-4033-94e2-e5046fe2079d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:58.887489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587a241-c1d0-4e2f-9100-ca4fd2b1fa63 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:58.916157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae633400-b14b-484a-8f25-e8f01a78736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:58.916826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae633400-b14b-484a-8f25-e8f01a78736d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:58.917112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae633400-b14b-484a-8f25-e8f01a78736d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:58.917431 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae633400-b14b-484a-8f25-e8f01a78736d None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:58.918195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae633400-b14b-484a-8f25-e8f01a78736d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:58.918615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae633400-b14b-484a-8f25-e8f01a78736d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:58.919851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae633400-b14b-484a-8f25-e8f01a78736d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:58.932205 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 422/850] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:14:58.937884 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae633400-b14b-484a-8f25-e8f01a78736d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:58.991125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:59.014360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5587a241-c1d0-4e2f-9100-ca4fd2b1fa63 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:59.015220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:59.051730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db3843-3c39-42b9-9b92-6ed1d519a2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:59.052614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db3843-3c39-42b9-9b92-6ed1d519a2bb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.052843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db3843-3c39-42b9-9b92-6ed1d519a2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.053119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db3843-3c39-42b9-9b92-6ed1d519a2bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.056641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae633400-b14b-484a-8f25-e8f01a78736d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:59.065416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db3843-3c39-42b9-9b92-6ed1d519a2bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-46db3843-3c39-42b9-9b92-6ed1d519a2bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:14:59.073505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:59.074440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:14:59.075274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:59.078740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 429/851] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:14:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:59.088796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db3843-3c39-42b9-9b92-6ed1d519a2bb None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:59.096189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59efd58-5244-4dec-b414-ca01adac1f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:59.097141 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59efd58-5244-4dec-b414-ca01adac1f12 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.098034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59efd58-5244-4dec-b414-ca01adac1f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.098034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59efd58-5244-4dec-b414-ca01adac1f12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.105633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59efd58-5244-4dec-b414-ca01adac1f12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:59.107296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59efd58-5244-4dec-b414-ca01adac1f12 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f59efd58-5244-4dec-b414-ca01adac1f12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:14:59.126061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:59.127612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:59.128718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:14:59.128718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:14:59.128718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:14:59.128718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:14:59.128718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:14:59.128718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:14:59.128718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:14:59.128718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:59.133669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 423/852] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:59.134953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59efd58-5244-4dec-b414-ca01adac1f12 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:59.220668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db3843-3c39-42b9-9b92-6ed1d519a2bb None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:59.249756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59efd58-5244-4dec-b414-ca01adac1f12 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:59.261228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2371e-7811-46da-b9ba-9ecb69f2fe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:59.262130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2371e-7811-46da-b9ba-9ecb69f2fe39 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.262566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2371e-7811-46da-b9ba-9ecb69f2fe39 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.262993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2371e-7811-46da-b9ba-9ecb69f2fe39 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.264013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2371e-7811-46da-b9ba-9ecb69f2fe39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:59.264535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2371e-7811-46da-b9ba-9ecb69f2fe39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:59.266211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a2371e-7811-46da-b9ba-9ecb69f2fe39 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:59.272089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:59.282164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:59.282398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 430/853] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:14:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:59.285230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2371e-7811-46da-b9ba-9ecb69f2fe39 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:59.287747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab6dd7-6784-480d-b837-ccc5d62e9905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:59.288762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab6dd7-6784-480d-b837-ccc5d62e9905 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.289412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab6dd7-6784-480d-b837-ccc5d62e9905 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.289668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab6dd7-6784-480d-b837-ccc5d62e9905 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.290484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab6dd7-6784-480d-b837-ccc5d62e9905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:59.290994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab6dd7-6784-480d-b837-ccc5d62e9905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:59.292458 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdab6dd7-6784-480d-b837-ccc5d62e9905 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:59.311771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 424/854] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:59.313170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab6dd7-6784-480d-b837-ccc5d62e9905 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:59.398561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a2371e-7811-46da-b9ba-9ecb69f2fe39 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:59.416006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab6dd7-6784-480d-b837-ccc5d62e9905 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:59.438133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e49e714-0728-4425-9c48-814ef84e9a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:59.439172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49e714-0728-4425-9c48-814ef84e9a1e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.439606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49e714-0728-4425-9c48-814ef84e9a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.440053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e49e714-0728-4425-9c48-814ef84e9a1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.451415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49e714-0728-4425-9c48-814ef84e9a1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:59.453289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e49e714-0728-4425-9c48-814ef84e9a1e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:59.454567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74f10e-b146-437c-ad3f-94f7719ee1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:59.455235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74f10e-b146-437c-ad3f-94f7719ee1cd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.455524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74f10e-b146-437c-ad3f-94f7719ee1cd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.455806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74f10e-b146-437c-ad3f-94f7719ee1cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.456886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74f10e-b146-437c-ad3f-94f7719ee1cd None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b69264fe173045e08c282171cd7370af) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:59.457210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74f10e-b146-437c-ad3f-94f7719ee1cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:59.459074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74f10e-b146-437c-ad3f-94f7719ee1cd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:59.469550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 425/855] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:14:59 2022] GET /identity/v3/role_assignments?user.id=b69264fe173045e08c282171cd7370af&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5e49e714-0728-4425-9c48-814ef84e9a1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:14:59.475068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:59.476111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:59.477033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:14:59.477033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:14:59.477033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:14:59.477033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:59.479451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 431/856] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:14:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:14:59.486702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74f10e-b146-437c-ad3f-94f7719ee1cd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:59.557159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:59.614915 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74f10e-b146-437c-ad3f-94f7719ee1cd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:59.623561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe25c64-5276-47d6-8e7f-bbcab27d220a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.623935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe25c64-5276-47d6-8e7f-bbcab27d220a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.624272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe25c64-5276-47d6-8e7f-bbcab27d220a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.626022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 432/857] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:14:59.631114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3cb3e-fd9f-4e1c-b447-7a195f8f87c3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.631407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3cb3e-fd9f-4e1c-b447-7a195f8f87c3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.631686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3cb3e-fd9f-4e1c-b447-7a195f8f87c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.657856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-26b3cb3e-fd9f-4e1c-b447-7a195f8f87c3 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:14:59.659083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-26b3cb3e-fd9f-4e1c-b447-7a195f8f87c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:14:59.659320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7f488-e663-455f-8701-03fce21993e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:59.660062 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7f488-e663-455f-8701-03fce21993e0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.660347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7f488-e663-455f-8701-03fce21993e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.660620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e7f488-e663-455f-8701-03fce21993e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.661666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7f488-e663-455f-8701-03fce21993e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:14:59.661977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7f488-e663-455f-8701-03fce21993e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:59.664193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e7f488-e663-455f-8701-03fce21993e0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:59.679604 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 426/858] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:14:59.684809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7f488-e663-455f-8701-03fce21993e0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:59.720155 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:59.753550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 433/859] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:14:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:14:59.774526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3cb3e-fd9f-4e1c-b447-7a195f8f87c3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:59.799787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e7f488-e663-455f-8701-03fce21993e0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:59.811083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:14:59.838842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879a6cc-2d04-4d71-983f-828cd5d6a8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:59.839663 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879a6cc-2d04-4d71-983f-828cd5d6a8c7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.839946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879a6cc-2d04-4d71-983f-828cd5d6a8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.840219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879a6cc-2d04-4d71-983f-828cd5d6a8c7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.847732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879a6cc-2d04-4d71-983f-828cd5d6a8c7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:59.849183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879a6cc-2d04-4d71-983f-828cd5d6a8c7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7879a6cc-2d04-4d71-983f-828cd5d6a8c7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:14:59.865236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:59.866238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:14:59.867792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 427/860] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:14:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:59.871974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879a6cc-2d04-4d71-983f-828cd5d6a8c7 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:59.896638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3cb3e-fd9f-4e1c-b447-7a195f8f87c3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:14:59.945283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1fc53-2025-4f54-9baa-f98ddaabe95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:14:59.946708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1fc53-2025-4f54-9baa-f98ddaabe95e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:14:59.946990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1fc53-2025-4f54-9baa-f98ddaabe95e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:14:59.947566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1fc53-2025-4f54-9baa-f98ddaabe95e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:14:59.948231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1fc53-2025-4f54-9baa-f98ddaabe95e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:14:59.950814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1fc53-2025-4f54-9baa-f98ddaabe95e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8af1fc53-2025-4f54-9baa-f98ddaabe95e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:14:59.966709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:14:59.969736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:14:59.971707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 434/861] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Jul 12 13:14:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:14:59.981797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1fc53-2025-4f54-9baa-f98ddaabe95e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:14:59.989424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879a6cc-2d04-4d71-983f-828cd5d6a8c7 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:00.026644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa602b73-efc0-4aea-9307-9c63534f6ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:00.027233 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa602b73-efc0-4aea-9307-9c63534f6ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:00.027642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa602b73-efc0-4aea-9307-9c63534f6ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:00.027917 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa602b73-efc0-4aea-9307-9c63534f6ec9 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:00.028763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa602b73-efc0-4aea-9307-9c63534f6ec9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:00.029069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa602b73-efc0-4aea-9307-9c63534f6ec9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:00.031393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa602b73-efc0-4aea-9307-9c63534f6ec9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:00.032553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:00.057417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 428/862] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 12 13:14:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:00.059099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa602b73-efc0-4aea-9307-9c63534f6ec9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:00.089207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:00.104165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1fc53-2025-4f54-9baa-f98ddaabe95e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:00.149284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bf166-a376-4f01-b705-eb7e9f1f0859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:00.150292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf166-a376-4f01-b705-eb7e9f1f0859 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:00.150754 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf166-a376-4f01-b705-eb7e9f1f0859 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:00.151504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf166-a376-4f01-b705-eb7e9f1f0859 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:00.152395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bf166-a376-4f01-b705-eb7e9f1f0859 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:00.152900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bf166-a376-4f01-b705-eb7e9f1f0859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:00.155008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bf166-a376-4f01-b705-eb7e9f1f0859 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:00.167197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 435/863] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:14:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:00.168814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bf166-a376-4f01-b705-eb7e9f1f0859 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:00.172909 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa602b73-efc0-4aea-9307-9c63534f6ec9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:00.208296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb6ee-0e49-4627-b579-111974da98a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:00.208948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb6ee-0e49-4627-b579-111974da98a9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:00.209237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb6ee-0e49-4627-b579-111974da98a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:00.209545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58abb6ee-0e49-4627-b579-111974da98a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:00.290843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abb6ee-0e49-4627-b579-111974da98a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:00.292222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58abb6ee-0e49-4627-b579-111974da98a9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:00.318531 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bf166-a376-4f01-b705-eb7e9f1f0859 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:00.321593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:00.335443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:00.368548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852e82f-2caf-49a9-aab6-4996956da712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:00.369847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852e82f-2caf-49a9-aab6-4996956da712 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:00.370610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852e82f-2caf-49a9-aab6-4996956da712 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:00.371117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852e82f-2caf-49a9-aab6-4996956da712 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:00.372892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852e82f-2caf-49a9-aab6-4996956da712 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:00.373427 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852e82f-2caf-49a9-aab6-4996956da712 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:00.375307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852e82f-2caf-49a9-aab6-4996956da712 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:00.389926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 436/864] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:00 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:00.416012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 429/865] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 13:15:00.422588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852e82f-2caf-49a9-aab6-4996956da712 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:00.539272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852e82f-2caf-49a9-aab6-4996956da712 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:00.577540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd6eb7-77b1-4e25-b243-5579c0717f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:00.578693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd6eb7-77b1-4e25-b243-5579c0717f79 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:00.579176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd6eb7-77b1-4e25-b243-5579c0717f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:00.579642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcd6eb7-77b1-4e25-b243-5579c0717f79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:00.591253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd6eb7-77b1-4e25-b243-5579c0717f79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:00.593605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcd6eb7-77b1-4e25-b243-5579c0717f79 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8dcd6eb7-77b1-4e25-b243-5579c0717f79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:00.611677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:00.612583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:00.613496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:00.613496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:00.613496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:00.613496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:00.615672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 437/866] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:00.620942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb6ee-0e49-4627-b579-111974da98a9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:00.735531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58abb6ee-0e49-4627-b579-111974da98a9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:00.764690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:00.773006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879a6ef8-5882-45e9-bf09-08603b30b740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:00.773932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6ef8-5882-45e9-bf09-08603b30b740 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:00.777583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6ef8-5882-45e9-bf09-08603b30b740 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:00.777583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879a6ef8-5882-45e9-bf09-08603b30b740 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:00.777583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a6ef8-5882-45e9-bf09-08603b30b740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:00.777583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a6ef8-5882-45e9-bf09-08603b30b740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:00.777583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879a6ef8-5882-45e9-bf09-08603b30b740 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:00.794743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 430/867] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:00.801050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd6eb7-77b1-4e25-b243-5579c0717f79 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:00.832360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:00.896182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68152b8-f021-4f76-9925-39ed44cd4c20 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:00.896529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68152b8-f021-4f76-9925-39ed44cd4c20 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:00.896752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68152b8-f021-4f76-9925-39ed44cd4c20 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:00.899353 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 431/868] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:00.907069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e105910-404c-4491-ab49-cc140a92f79d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:00.907350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e105910-404c-4491-ab49-cc140a92f79d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:00.907637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e105910-404c-4491-ab49-cc140a92f79d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:00.926685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcd6eb7-77b1-4e25-b243-5579c0717f79 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:00.930875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5e105910-404c-4491-ab49-cc140a92f79d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:00.932079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5e105910-404c-4491-ab49-cc140a92f79d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:00.968257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccde0a-5528-494b-a9cd-ed5ef5a8cb4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:00.969242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccde0a-5528-494b-a9cd-ed5ef5a8cb4f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:00.969551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccde0a-5528-494b-a9cd-ed5ef5a8cb4f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:00.969917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ccde0a-5528-494b-a9cd-ed5ef5a8cb4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:00.977746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ccde0a-5528-494b-a9cd-ed5ef5a8cb4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:00.979599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ccde0a-5528-494b-a9cd-ed5ef5a8cb4f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-99ccde0a-5528-494b-a9cd-ed5ef5a8cb4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:15:00.998576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:00.999536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:01.000419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:15:01.000419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:15:01.000419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:15:01.000419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:15:01.000419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:15:01.000419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:15:01.000419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:01.000419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:01.000852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 438/869] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:01.006106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccde0a-5528-494b-a9cd-ed5ef5a8cb4f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:01.021981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 432/870] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:00 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:01.039610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e105910-404c-4491-ab49-cc140a92f79d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:01.104672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:01.122742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ccde0a-5528-494b-a9cd-ed5ef5a8cb4f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:01.164473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbf52b-11a3-4a06-9c5f-ad18339b9c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:01.165243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbf52b-11a3-4a06-9c5f-ad18339b9c39 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:01.165639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbf52b-11a3-4a06-9c5f-ad18339b9c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:01.165993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbf52b-11a3-4a06-9c5f-ad18339b9c39 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:01.166926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbf52b-11a3-4a06-9c5f-ad18339b9c39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:01.167312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbf52b-11a3-4a06-9c5f-ad18339b9c39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:01.168855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbf52b-11a3-4a06-9c5f-ad18339b9c39 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:01.173778 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e105910-404c-4491-ab49-cc140a92f79d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:01.185741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 439/871] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:15:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:01.189085 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbf52b-11a3-4a06-9c5f-ad18339b9c39 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:01.194829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:01.216754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f371ce-cf3d-4522-a99d-069e6cf86dfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:01.217452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f371ce-cf3d-4522-a99d-069e6cf86dfb None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:01.218430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f371ce-cf3d-4522-a99d-069e6cf86dfb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:01.218717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f371ce-cf3d-4522-a99d-069e6cf86dfb None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:01.219713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f371ce-cf3d-4522-a99d-069e6cf86dfb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:01.221563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f371ce-cf3d-4522-a99d-069e6cf86dfb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:01.255758 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 433/872] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:15:01 2022] POST /identity/v3/services => generated 251 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:15:01.324996 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbf52b-11a3-4a06-9c5f-ad18339b9c39 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:01.370981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec5243d-67ea-4197-8d6a-ced94ac63b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:01.371894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5243d-67ea-4197-8d6a-ced94ac63b90 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:01.372355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5243d-67ea-4197-8d6a-ced94ac63b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:01.372775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec5243d-67ea-4197-8d6a-ced94ac63b90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:01.373851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5243d-67ea-4197-8d6a-ced94ac63b90 None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b26fa3bc9c0d4e22a8abc5a4d98c9995) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:01.374488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5243d-67ea-4197-8d6a-ced94ac63b90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:01.376098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec5243d-67ea-4197-8d6a-ced94ac63b90 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:01.383520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 440/873] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:01 2022] GET /identity/v3/role_assignments?user.id=b26fa3bc9c0d4e22a8abc5a4d98c9995&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:01.673419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901042df-c043-4ac1-902d-51ebb2e51fc6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:01.673768 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901042df-c043-4ac1-902d-51ebb2e51fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:01.673958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901042df-c043-4ac1-902d-51ebb2e51fc6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:01.675523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 434/874] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:01.682934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225581c7-560d-47ad-b195-f01f98e11a92 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:01.682934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225581c7-560d-47ad-b195-f01f98e11a92 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:01.682934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225581c7-560d-47ad-b195-f01f98e11a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:01.694224 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:01.700203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-225581c7-560d-47ad-b195-f01f98e11a92 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:01.701696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-225581c7-560d-47ad-b195-f01f98e11a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:01.777456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 441/875] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:01 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:01.797618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901042df-c043-4ac1-902d-51ebb2e51fc6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:01.910941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901042df-c043-4ac1-902d-51ebb2e51fc6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:01.949006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40ba6f-c601-4ebe-83de-9d9d899ab639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:01.949783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ba6f-c601-4ebe-83de-9d9d899ab639 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:01.950120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ba6f-c601-4ebe-83de-9d9d899ab639 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:01.950403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad40ba6f-c601-4ebe-83de-9d9d899ab639 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:01.951072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40ba6f-c601-4ebe-83de-9d9d899ab639 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:01.952985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad40ba6f-c601-4ebe-83de-9d9d899ab639 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ad40ba6f-c601-4ebe-83de-9d9d899ab639 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:01.966225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:01.967245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:01.969286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 435/876] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:01.976468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225581c7-560d-47ad-b195-f01f98e11a92 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:02.089181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225581c7-560d-47ad-b195-f01f98e11a92 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:02.127042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6722ae8-dd53-4a79-9abb-d806904283c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:02.128079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6722ae8-dd53-4a79-9abb-d806904283c0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:02.128572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6722ae8-dd53-4a79-9abb-d806904283c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:02.129053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6722ae8-dd53-4a79-9abb-d806904283c0 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:02.130057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6722ae8-dd53-4a79-9abb-d806904283c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:02.130572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6722ae8-dd53-4a79-9abb-d806904283c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:02.132163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6722ae8-dd53-4a79-9abb-d806904283c0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:02.134745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:02.143289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 442/877] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:02.145090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40ba6f-c601-4ebe-83de-9d9d899ab639 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:02.192654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:02.263481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad40ba6f-c601-4ebe-83de-9d9d899ab639 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:02.298385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910d063b-85c8-4e7f-a753-9faaccb367c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:02.299056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d063b-85c8-4e7f-a753-9faaccb367c4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:02.299342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d063b-85c8-4e7f-a753-9faaccb367c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:02.299621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910d063b-85c8-4e7f-a753-9faaccb367c4 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:02.300337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d063b-85c8-4e7f-a753-9faaccb367c4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:02.300644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d063b-85c8-4e7f-a753-9faaccb367c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:02.302040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910d063b-85c8-4e7f-a753-9faaccb367c4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:02.306962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 436/878] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:02.527597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2eb5d-b6e3-407f-b595-58add882b466 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:02.527940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2eb5d-b6e3-407f-b595-58add882b466 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:02.528120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b2eb5d-b6e3-407f-b595-58add882b466 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:02.529676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 443/879] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:02.535819 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28a348-852f-4f32-9ca6-d3ddbee901fa None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:02.536109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28a348-852f-4f32-9ca6-d3ddbee901fa None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:02.536392 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28a348-852f-4f32-9ca6-d3ddbee901fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:02.553647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cf28a348-852f-4f32-9ca6-d3ddbee901fa None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:02.554850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cf28a348-852f-4f32-9ca6-d3ddbee901fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:02.626902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 437/880] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:02.649798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2eb5d-b6e3-407f-b595-58add882b466 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:02.753542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b2eb5d-b6e3-407f-b595-58add882b466 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:02.788998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a5ded-7b30-432d-b9a5-c11f156ccd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:02.789623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a5ded-7b30-432d-b9a5-c11f156ccd64 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:02.789901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a5ded-7b30-432d-b9a5-c11f156ccd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:02.790184 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a5ded-7b30-432d-b9a5-c11f156ccd64 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:02.790915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a5ded-7b30-432d-b9a5-c11f156ccd64 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:02.792273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a5ded-7b30-432d-b9a5-c11f156ccd64 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:02.810272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 444/881] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:15:02 2022] POST /identity/v3/services => generated 269 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:03.461217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cff89-670a-48ca-b7cc-48da46bc74d8 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:03.461761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cff89-670a-48ca-b7cc-48da46bc74d8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:03.461839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6cff89-670a-48ca-b7cc-48da46bc74d8 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:03.463447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 438/882] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:03.469681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd49ee2-f35b-4a9d-8f8b-362482f1eb7b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:03.470122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd49ee2-f35b-4a9d-8f8b-362482f1eb7b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:03.470297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd49ee2-f35b-4a9d-8f8b-362482f1eb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:03.487824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8cd49ee2-f35b-4a9d-8f8b-362482f1eb7b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:03.489040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8cd49ee2-f35b-4a9d-8f8b-362482f1eb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:03.550342 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:03.568793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 445/883] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:03.592597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cff89-670a-48ca-b7cc-48da46bc74d8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:03.627131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f7a4d-f49c-455f-ab48-cfa7fc442257 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:03.627475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f7a4d-f49c-455f-ab48-cfa7fc442257 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:03.627763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f7a4d-f49c-455f-ab48-cfa7fc442257 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:03.629119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 446/884] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:03.634214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3822ce-aefd-45fb-8f38-388b44b41357 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:03.634509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3822ce-aefd-45fb-8f38-388b44b41357 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:03.634786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3822ce-aefd-45fb-8f38-388b44b41357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:03.636351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:03.657688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2f3822ce-aefd-45fb-8f38-388b44b41357 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:03.659864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2f3822ce-aefd-45fb-8f38-388b44b41357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:03.710584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6cff89-670a-48ca-b7cc-48da46bc74d8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:03.741527 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 447/885] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:03.754430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0506280-d59e-479f-8aeb-df3b8315fc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:03.755242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0506280-d59e-479f-8aeb-df3b8315fc2f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:03.755582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0506280-d59e-479f-8aeb-df3b8315fc2f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:03.755870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0506280-d59e-479f-8aeb-df3b8315fc2f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:03.756534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0506280-d59e-479f-8aeb-df3b8315fc2f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:03.757948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0506280-d59e-479f-8aeb-df3b8315fc2f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:03.763740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3822ce-aefd-45fb-8f38-388b44b41357 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f0506280-d59e-479f-8aeb-df3b8315fc2f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:03.772514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:03.773781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:03.774638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 439/886] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:15:03 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:03.779816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0506280-d59e-479f-8aeb-df3b8315fc2f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:03.883305 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:03.886950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0506280-d59e-479f-8aeb-df3b8315fc2f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:03.897532 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3822ce-aefd-45fb-8f38-388b44b41357 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:03.916641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:03.925120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526b227-6e16-4abb-bc09-6dc68d188550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:03.925844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526b227-6e16-4abb-bc09-6dc68d188550 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:03.926325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526b227-6e16-4abb-bc09-6dc68d188550 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:03.926753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526b227-6e16-4abb-bc09-6dc68d188550 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:03.927618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526b227-6e16-4abb-bc09-6dc68d188550 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:03.928074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526b227-6e16-4abb-bc09-6dc68d188550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:03.929622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526b227-6e16-4abb-bc09-6dc68d188550 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:03.935217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 440/887] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:15:03 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:03.938052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c24bce-ea61-407e-a404-fbba7218a714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:03.938879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c24bce-ea61-407e-a404-fbba7218a714 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:03.939163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c24bce-ea61-407e-a404-fbba7218a714 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:03.939437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c24bce-ea61-407e-a404-fbba7218a714 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:03.941991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526b227-6e16-4abb-bc09-6dc68d188550 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:03.951292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c24bce-ea61-407e-a404-fbba7218a714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-53c24bce-ea61-407e-a404-fbba7218a714 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:03.960658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:03.961782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:03.962608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:03.967187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 448/888] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 12 13:15:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:03.968171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c24bce-ea61-407e-a404-fbba7218a714 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:04.055495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526b227-6e16-4abb-bc09-6dc68d188550 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:04.074691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c24bce-ea61-407e-a404-fbba7218a714 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:04.093791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b8732-8645-4d72-9a6b-76121c6f1f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:04.094841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b8732-8645-4d72-9a6b-76121c6f1f14 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.095326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b8732-8645-4d72-9a6b-76121c6f1f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.095805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b8732-8645-4d72-9a6b-76121c6f1f14 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.096826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b8732-8645-4d72-9a6b-76121c6f1f14 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:04.097334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b8732-8645-4d72-9a6b-76121c6f1f14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:04.098953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1b8732-8645-4d72-9a6b-76121c6f1f14 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:04.105063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 441/889] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:15:03 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:04.112882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289753dc-df98-454e-b2de-f28b6c815933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:04.113591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289753dc-df98-454e-b2de-f28b6c815933 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.113880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289753dc-df98-454e-b2de-f28b6c815933 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.114160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289753dc-df98-454e-b2de-f28b6c815933 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.114912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289753dc-df98-454e-b2de-f28b6c815933 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:04.115195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b8732-8645-4d72-9a6b-76121c6f1f14 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:04.115301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289753dc-df98-454e-b2de-f28b6c815933 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:04.116578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289753dc-df98-454e-b2de-f28b6c815933 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:04.129466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 449/890] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 12 13:15:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:04.131968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289753dc-df98-454e-b2de-f28b6c815933 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:04.147826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:04.178351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:04.229014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b8732-8645-4d72-9a6b-76121c6f1f14 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:04.261841 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289753dc-df98-454e-b2de-f28b6c815933 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:04.265406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3aff6-d4f8-4ee6-a988-bf48bdff22dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:04.266121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3aff6-d4f8-4ee6-a988-bf48bdff22dd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.266477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3aff6-d4f8-4ee6-a988-bf48bdff22dd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.266823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a3aff6-d4f8-4ee6-a988-bf48bdff22dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.267691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3aff6-d4f8-4ee6-a988-bf48bdff22dd None admin] RBAC: Adding query filter params (interface=public, service_id=cacc40d3d02a42bf9625a98ff32f5429, region_id=RegionOne) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:04.268082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3aff6-d4f8-4ee6-a988-bf48bdff22dd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:04.269580 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a3aff6-d4f8-4ee6-a988-bf48bdff22dd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:04.275437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 442/891] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 12 13:15:04 2022] GET /identity/v3/endpoints?service_id=cacc40d3d02a42bf9625a98ff32f5429&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:04.280944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3aff6-d4f8-4ee6-a988-bf48bdff22dd None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:04.308147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b402ab-6505-4f1b-9160-1579fa0eca60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:04.309680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b402ab-6505-4f1b-9160-1579fa0eca60 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.310415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b402ab-6505-4f1b-9160-1579fa0eca60 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.311597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b402ab-6505-4f1b-9160-1579fa0eca60 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.326186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b402ab-6505-4f1b-9160-1579fa0eca60 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:04.329291 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b402ab-6505-4f1b-9160-1579fa0eca60 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-49b402ab-6505-4f1b-9160-1579fa0eca60 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:04.354667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:04.355601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:04.356491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:04.356491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:04.356491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:04.356491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:04.363807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 450/892] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:04.364526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b402ab-6505-4f1b-9160-1579fa0eca60 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:04.397476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:04.409350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a3aff6-d4f8-4ee6-a988-bf48bdff22dd None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:04.446114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a6b01a-f030-45e4-b965-7f535a74e95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:04.446777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6b01a-f030-45e4-b965-7f535a74e95f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.447061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6b01a-f030-45e4-b965-7f535a74e95f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.447340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a6b01a-f030-45e4-b965-7f535a74e95f None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.448026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6b01a-f030-45e4-b965-7f535a74e95f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:04.449407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a6b01a-f030-45e4-b965-7f535a74e95f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:04.455779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 443/893] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 12 13:15:04 2022] GET /identity/v3/services => generated 1268 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:15:04.457083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c075ed-4668-4e2c-8ba0-3b946bccea62 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.457390 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c075ed-4668-4e2c-8ba0-3b946bccea62 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.457666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c075ed-4668-4e2c-8ba0-3b946bccea62 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.459103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 444/894] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:04.464627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4797d-62d6-4aae-8979-70b301abb5e1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.465242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4797d-62d6-4aae-8979-70b301abb5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.465242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4797d-62d6-4aae-8979-70b301abb5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.482410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:04.484773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-18f4797d-62d6-4aae-8979-70b301abb5e1 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:04.485982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-18f4797d-62d6-4aae-8979-70b301abb5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:04.496388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b402ab-6505-4f1b-9160-1579fa0eca60 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:04.543177 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b88baa-bc3e-40d4-a47f-b23bbe57547e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:04.544087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88baa-bc3e-40d4-a47f-b23bbe57547e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.544771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88baa-bc3e-40d4-a47f-b23bbe57547e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.545117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b88baa-bc3e-40d4-a47f-b23bbe57547e None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.546676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b88baa-bc3e-40d4-a47f-b23bbe57547e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:04.547055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b88baa-bc3e-40d4-a47f-b23bbe57547e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:04.549112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b88baa-bc3e-40d4-a47f-b23bbe57547e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:04.564526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 451/895] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:04.570476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b88baa-bc3e-40d4-a47f-b23bbe57547e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:04.573771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 445/896] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:04.591821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4797d-62d6-4aae-8979-70b301abb5e1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:04.689876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:04.693756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b88baa-bc3e-40d4-a47f-b23bbe57547e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:04.709414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4797d-62d6-4aae-8979-70b301abb5e1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:04.738699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a957c-b333-4019-b02d-b8aa331e367f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:04.740394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a957c-b333-4019-b02d-b8aa331e367f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.741127 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a957c-b333-4019-b02d-b8aa331e367f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.741586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514a957c-b333-4019-b02d-b8aa331e367f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.751800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a957c-b333-4019-b02d-b8aa331e367f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:04.752655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93943e91-cfa4-4c9e-ba7b-f38e8f561717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:04.753325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93943e91-cfa4-4c9e-ba7b-f38e8f561717 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.753818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514a957c-b333-4019-b02d-b8aa331e367f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:04.753937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93943e91-cfa4-4c9e-ba7b-f38e8f561717 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.754180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93943e91-cfa4-4c9e-ba7b-f38e8f561717 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.755104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93943e91-cfa4-4c9e-ba7b-f38e8f561717 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:04.756809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93943e91-cfa4-4c9e-ba7b-f38e8f561717 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-514a957c-b333-4019-b02d-b8aa331e367f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:15:04.772763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:04.774058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:15:04.775220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:04.777412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 452/898] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:15:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:04.781577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 446/898] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:15:04 2022] POST /identity/v3/services => generated 241 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:04.787700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a957c-b333-4019-b02d-b8aa331e367f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:04.810878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:04.902154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514a957c-b333-4019-b02d-b8aa331e367f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:04.916967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dbb96-5573-4c20-a51b-62b4dabe694d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.917306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dbb96-5573-4c20-a51b-62b4dabe694d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.917561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1dbb96-5573-4c20-a51b-62b4dabe694d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.919207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 447/899] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:04.926882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0377d9-bffe-44a1-a682-e42ab689824e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.927175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0377d9-bffe-44a1-a682-e42ab689824e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.927497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0377d9-bffe-44a1-a682-e42ab689824e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.953690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3a0377d9-bffe-44a1-a682-e42ab689824e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:04.955671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3a0377d9-bffe-44a1-a682-e42ab689824e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:04.956358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835b78b-6a53-4bc8-8723-9135d0f71a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:04.957100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b78b-6a53-4bc8-8723-9135d0f71a98 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:04.957412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b78b-6a53-4bc8-8723-9135d0f71a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:04.957687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7835b78b-6a53-4bc8-8723-9135d0f71a98 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:04.958547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835b78b-6a53-4bc8-8723-9135d0f71a98 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:04.958850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835b78b-6a53-4bc8-8723-9135d0f71a98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:04.960534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7835b78b-6a53-4bc8-8723-9135d0f71a98 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:04.986710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 453/900] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 12 13:15:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:04.992835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835b78b-6a53-4bc8-8723-9135d0f71a98 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:05.043841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 448/901] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:05.067070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0377d9-bffe-44a1-a682-e42ab689824e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:05.082551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:05.141128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7835b78b-6a53-4bc8-8723-9135d0f71a98 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:05.156575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:05.185855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0377d9-bffe-44a1-a682-e42ab689824e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:05.198539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7841a5-fce6-4948-95d8-3e320cc2575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:05.199973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7841a5-fce6-4948-95d8-3e320cc2575a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:05.200750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7841a5-fce6-4948-95d8-3e320cc2575a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:05.201240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7841a5-fce6-4948-95d8-3e320cc2575a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:05.234644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4b515-4607-4d0f-afde-f3a79f88fa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:05.235489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4b515-4607-4d0f-afde-f3a79f88fa67 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:05.235773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4b515-4607-4d0f-afde-f3a79f88fa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:05.236311 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b4b515-4607-4d0f-afde-f3a79f88fa67 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:05.237499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4b515-4607-4d0f-afde-f3a79f88fa67 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:05.239271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b4b515-4607-4d0f-afde-f3a79f88fa67 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-44b4b515-4607-4d0f-afde-f3a79f88fa67 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:05.257338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:05.259235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:15:05.260568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:05.263829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 449/902] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Jul 12 13:15:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:05.266507 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4b515-4607-4d0f-afde-f3a79f88fa67 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:05.306083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7841a5-fce6-4948-95d8-3e320cc2575a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:05.309029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7841a5-fce6-4948-95d8-3e320cc2575a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:05.379943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b4b515-4607-4d0f-afde-f3a79f88fa67 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:05.391210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:05.426700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ad7e5-2eb3-4fe8-b785-cc971bc96481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:05.427526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ad7e5-2eb3-4fe8-b785-cc971bc96481 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:05.427927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ad7e5-2eb3-4fe8-b785-cc971bc96481 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:05.428819 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0ad7e5-2eb3-4fe8-b785-cc971bc96481 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:05.429703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ad7e5-2eb3-4fe8-b785-cc971bc96481 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:05.430393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ad7e5-2eb3-4fe8-b785-cc971bc96481 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:05.432133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0ad7e5-2eb3-4fe8-b785-cc971bc96481 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:05.441409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 450/903] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:05.445743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ad7e5-2eb3-4fe8-b785-cc971bc96481 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:05.447247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 454/904] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:04 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:05.449972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:05.453152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7841a5-fce6-4948-95d8-3e320cc2575a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:05.559500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7841a5-fce6-4948-95d8-3e320cc2575a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:05.588187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0ad7e5-2eb3-4fe8-b785-cc971bc96481 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:05.597399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e0f32-12c5-482e-9a59-033e918ce3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:05.598619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e0f32-12c5-482e-9a59-033e918ce3ab None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:05.598788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e0f32-12c5-482e-9a59-033e918ce3ab None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:05.599288 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359e0f32-12c5-482e-9a59-033e918ce3ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:05.610751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e0f32-12c5-482e-9a59-033e918ce3ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:05.612821 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359e0f32-12c5-482e-9a59-033e918ce3ab None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:05.629718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e221d8d-a270-41bb-b7e5-3b3af268bbe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:05.630389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e221d8d-a270-41bb-b7e5-3b3af268bbe1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:05.630678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e221d8d-a270-41bb-b7e5-3b3af268bbe1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:05.630961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e221d8d-a270-41bb-b7e5-3b3af268bbe1 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:05.631684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e221d8d-a270-41bb-b7e5-3b3af268bbe1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-359e0f32-12c5-482e-9a59-033e918ce3ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:05.631861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:05.633818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:05.635918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:05.635918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:05.635918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:05.635918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:05.636426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e221d8d-a270-41bb-b7e5-3b3af268bbe1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:05.636426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e221d8d-a270-41bb-b7e5-3b3af268bbe1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:05.641418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 455/905] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:05.641418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e0f32-12c5-482e-9a59-033e918ce3ab None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:05.641662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 451/906] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:05.651405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e221d8d-a270-41bb-b7e5-3b3af268bbe1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:05.656956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:05.727982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:05.746865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359e0f32-12c5-482e-9a59-033e918ce3ab None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:05.767113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e221d8d-a270-41bb-b7e5-3b3af268bbe1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:05.783777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034a40f-93b0-44fc-81dc-64f58893928a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:05.784461 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034a40f-93b0-44fc-81dc-64f58893928a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:05.784742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034a40f-93b0-44fc-81dc-64f58893928a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:05.785017 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c034a40f-93b0-44fc-81dc-64f58893928a None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:05.785876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034a40f-93b0-44fc-81dc-64f58893928a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:05.786184 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034a40f-93b0-44fc-81dc-64f58893928a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:05.787649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c034a40f-93b0-44fc-81dc-64f58893928a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:05.804522 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 456/907] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:05.806173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034a40f-93b0-44fc-81dc-64f58893928a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:05.808197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fe926-65ff-4cf7-b110-cb99899fdfe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:05.808902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fe926-65ff-4cf7-b110-cb99899fdfe9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:05.809188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fe926-65ff-4cf7-b110-cb99899fdfe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:05.809480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6fe926-65ff-4cf7-b110-cb99899fdfe9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:05.810514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fe926-65ff-4cf7-b110-cb99899fdfe9 None admin] RBAC: Adding query filter params (interface=public, service_id=1cc55c98f2f14f0d944b1a18b4cb61fa, region_id=RegionOne) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:05.810774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fe926-65ff-4cf7-b110-cb99899fdfe9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:05.813152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6fe926-65ff-4cf7-b110-cb99899fdfe9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:05.822903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 452/908] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 12 13:15:05 2022] GET /identity/v3/endpoints?service_id=1cc55c98f2f14f0d944b1a18b4cb61fa&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:05.824548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fe926-65ff-4cf7-b110-cb99899fdfe9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:05.899154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:05.926333 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c034a40f-93b0-44fc-81dc-64f58893928a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:05.961956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6fe926-65ff-4cf7-b110-cb99899fdfe9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:05.963233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38aa1b-97a5-400b-8cb0-8140e47e67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:05.964169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38aa1b-97a5-400b-8cb0-8140e47e67b9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:05.964460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38aa1b-97a5-400b-8cb0-8140e47e67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:05.964740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad38aa1b-97a5-400b-8cb0-8140e47e67b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:05.972756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38aa1b-97a5-400b-8cb0-8140e47e67b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:05.974392 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad38aa1b-97a5-400b-8cb0-8140e47e67b9 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ad38aa1b-97a5-400b-8cb0-8140e47e67b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:15:05.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:05.991798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:05.993110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:15:05.993110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:15:05.993110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:15:05.993110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:15:05.993110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:15:05.993110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:15:05.993110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:05.993110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:05.993110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 457/909] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:05.997354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38aa1b-97a5-400b-8cb0-8140e47e67b9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:06.004666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:06.011672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d2b35-e47a-4387-b4c6-e2cfb11aeaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:06.012732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d2b35-e47a-4387-b4c6-e2cfb11aeaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:06.013249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d2b35-e47a-4387-b4c6-e2cfb11aeaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:06.013779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d2b35-e47a-4387-b4c6-e2cfb11aeaf0 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:06.014934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d2b35-e47a-4387-b4c6-e2cfb11aeaf0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:06.017099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d2b35-e47a-4387-b4c6-e2cfb11aeaf0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:06.024209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 453/910] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 12 13:15:05 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:15:06.117570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad38aa1b-97a5-400b-8cb0-8140e47e67b9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:06.155615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7b9152-0f90-49b3-9e60-28c14dcb4440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:06.156304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b9152-0f90-49b3-9e60-28c14dcb4440 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:06.156589 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b9152-0f90-49b3-9e60-28c14dcb4440 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:06.156869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7b9152-0f90-49b3-9e60-28c14dcb4440 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:06.157710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b9152-0f90-49b3-9e60-28c14dcb4440 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:06.158024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b9152-0f90-49b3-9e60-28c14dcb4440 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:06.159484 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7b9152-0f90-49b3-9e60-28c14dcb4440 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:06.162545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:06.173831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 458/911] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:15:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:06.179640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d2b35-e47a-4387-b4c6-e2cfb11aeaf0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:06.288658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d2b35-e47a-4387-b4c6-e2cfb11aeaf0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:06.324338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a60be95-4a6d-4d58-a9ab-bea35b2232ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:06.325003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60be95-4a6d-4d58-a9ab-bea35b2232ae None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:06.325295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60be95-4a6d-4d58-a9ab-bea35b2232ae None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:06.325605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a60be95-4a6d-4d58-a9ab-bea35b2232ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:06.326448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60be95-4a6d-4d58-a9ab-bea35b2232ae None admin] RBAC: Adding query filter params (role.id=4c2749d472714b3c85b29f3732da518e, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b26fa3bc9c0d4e22a8abc5a4d98c9995) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:06.326765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60be95-4a6d-4d58-a9ab-bea35b2232ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:06.328797 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a60be95-4a6d-4d58-a9ab-bea35b2232ae None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:06.334798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 454/912] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:06 2022] GET /identity/v3/role_assignments?user.id=b26fa3bc9c0d4e22a8abc5a4d98c9995&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=4c2749d472714b3c85b29f3732da518e => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:06.616906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a475e4-48d5-4f97-8f7c-0c058478c710 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:06.617635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a475e4-48d5-4f97-8f7c-0c058478c710 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:06.617635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a475e4-48d5-4f97-8f7c-0c058478c710 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:06.618967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 459/913] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:06.624362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674072c-d290-43a1-b892-2ae73cc06285 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:06.624896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674072c-d290-43a1-b892-2ae73cc06285 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:06.624896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674072c-d290-43a1-b892-2ae73cc06285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:06.643132 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e674072c-d290-43a1-b892-2ae73cc06285 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:06.644334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e674072c-d290-43a1-b892-2ae73cc06285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:06.704842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:06.718163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 455/914] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:06.740869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a475e4-48d5-4f97-8f7c-0c058478c710 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:06.853142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a475e4-48d5-4f97-8f7c-0c058478c710 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:06.888663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7539f8-eb53-4c9d-a32f-f25522e7fbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:06.889453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7539f8-eb53-4c9d-a32f-f25522e7fbee None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:06.889742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7539f8-eb53-4c9d-a32f-f25522e7fbee None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:06.890018 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7539f8-eb53-4c9d-a32f-f25522e7fbee None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:06.890792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7539f8-eb53-4c9d-a32f-f25522e7fbee None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:06.892161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7539f8-eb53-4c9d-a32f-f25522e7fbee None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2b7539f8-eb53-4c9d-a32f-f25522e7fbee None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:06.904468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:06.906059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:06.906882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 460/915] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:15:06 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:06.914323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674072c-d290-43a1-b892-2ae73cc06285 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:06.965881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26eb47-7edc-41b8-945a-fa63ba2dd402 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:06.966207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26eb47-7edc-41b8-945a-fa63ba2dd402 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:06.966446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e26eb47-7edc-41b8-945a-fa63ba2dd402 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:06.968703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 461/916] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:06.973867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bceea-528f-4828-847f-fd0b6cc3f33f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:06.974163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bceea-528f-4828-847f-fd0b6cc3f33f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:06.974448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bceea-528f-4828-847f-fd0b6cc3f33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:06.992664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2e5bceea-528f-4828-847f-fd0b6cc3f33f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:06.993899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2e5bceea-528f-4828-847f-fd0b6cc3f33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:07.023452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674072c-d290-43a1-b892-2ae73cc06285 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:07.055815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:07.064735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a98b1-635f-445a-9fcb-88a3911f6f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:07.066355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a98b1-635f-445a-9fcb-88a3911f6f48 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:07.066653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a98b1-635f-445a-9fcb-88a3911f6f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:07.066936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a98b1-635f-445a-9fcb-88a3911f6f48 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:07.067665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a98b1-635f-445a-9fcb-88a3911f6f48 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:07.067985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a98b1-635f-445a-9fcb-88a3911f6f48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:07.069363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a98b1-635f-445a-9fcb-88a3911f6f48 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:07.070783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 462/917] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:07.082038 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 456/918] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:15:06 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:07.089397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bceea-528f-4828-847f-fd0b6cc3f33f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:07.096775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a98b1-635f-445a-9fcb-88a3911f6f48 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:07.169680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:07.200050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bceea-528f-4828-847f-fd0b6cc3f33f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:07.222879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a98b1-635f-445a-9fcb-88a3911f6f48 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:07.239803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260ec74-da28-4c8f-86f6-90e45cb0209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:07.240560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260ec74-da28-4c8f-86f6-90e45cb0209f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:07.240768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260ec74-da28-4c8f-86f6-90e45cb0209f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:07.241045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4260ec74-da28-4c8f-86f6-90e45cb0209f None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:07.241805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260ec74-da28-4c8f-86f6-90e45cb0209f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:07.242119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260ec74-da28-4c8f-86f6-90e45cb0209f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:07.243507 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4260ec74-da28-4c8f-86f6-90e45cb0209f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:07.253851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 463/919] 10.222.0.23 () {56 vars in 1114 bytes} [Tue Jul 12 13:15:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:07.255100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260ec74-da28-4c8f-86f6-90e45cb0209f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:07.271572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f1808-f608-40c7-8bf6-c6d2198563be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:07.272391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f1808-f608-40c7-8bf6-c6d2198563be None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:07.272682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f1808-f608-40c7-8bf6-c6d2198563be None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:07.272963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064f1808-f608-40c7-8bf6-c6d2198563be None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:07.273814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f1808-f608-40c7-8bf6-c6d2198563be None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:07.275838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064f1808-f608-40c7-8bf6-c6d2198563be None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-064f1808-f608-40c7-8bf6-c6d2198563be None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:07.291850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:07.292844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:07.294061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:07.297466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 457/920] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:07.301067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f1808-f608-40c7-8bf6-c6d2198563be None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:07.327226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:07.372453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4260ec74-da28-4c8f-86f6-90e45cb0209f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:07.408385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a4530-7b3a-42eb-a1a7-39a8c0ada923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:07.409421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a4530-7b3a-42eb-a1a7-39a8c0ada923 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:07.409918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a4530-7b3a-42eb-a1a7-39a8c0ada923 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:07.410399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a4530-7b3a-42eb-a1a7-39a8c0ada923 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:07.411358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a4530-7b3a-42eb-a1a7-39a8c0ada923 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:07.412523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064f1808-f608-40c7-8bf6-c6d2198563be None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:07.413209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a4530-7b3a-42eb-a1a7-39a8c0ada923 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:07.431521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:07.435388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 464/921] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 12 13:15:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:15:07.465531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b8ed5-8209-483d-b0a7-3d950325a38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:07.466683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b8ed5-8209-483d-b0a7-3d950325a38f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:07.467282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b8ed5-8209-483d-b0a7-3d950325a38f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:07.468600 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75b8ed5-8209-483d-b0a7-3d950325a38f None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:07.469673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b8ed5-8209-483d-b0a7-3d950325a38f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:07.470214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b8ed5-8209-483d-b0a7-3d950325a38f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:07.471975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75b8ed5-8209-483d-b0a7-3d950325a38f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:07.483695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 458/922] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:07.486310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a4530-7b3a-42eb-a1a7-39a8c0ada923 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:07.611434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a4530-7b3a-42eb-a1a7-39a8c0ada923 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:07.622225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:07.651896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf849e-7e46-4e42-a148-05e2c8b8da6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:07.652576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf849e-7e46-4e42-a148-05e2c8b8da6e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:07.652865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf849e-7e46-4e42-a148-05e2c8b8da6e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:07.653145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baf849e-7e46-4e42-a148-05e2c8b8da6e None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:07.653904 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf849e-7e46-4e42-a148-05e2c8b8da6e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:07.654218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf849e-7e46-4e42-a148-05e2c8b8da6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:07.655757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baf849e-7e46-4e42-a148-05e2c8b8da6e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:07.661058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 465/923] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:07.667384 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b8ed5-8209-483d-b0a7-3d950325a38f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:07.773234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75b8ed5-8209-483d-b0a7-3d950325a38f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:07.807429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae08ffa-8712-4290-becb-47610568541e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:07.808094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae08ffa-8712-4290-becb-47610568541e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:07.808376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae08ffa-8712-4290-becb-47610568541e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:07.808659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae08ffa-8712-4290-becb-47610568541e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:07.809499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae08ffa-8712-4290-becb-47610568541e None admin] RBAC: Adding query filter params (interface=public, service_id=bacb41317bc449868db96e1ee7d1675b, region_id=RegionOne) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:07.809815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae08ffa-8712-4290-becb-47610568541e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:07.811724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae08ffa-8712-4290-becb-47610568541e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:07.817367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 459/924] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 12 13:15:07 2022] GET /identity/v3/endpoints?service_id=bacb41317bc449868db96e1ee7d1675b&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:07.823361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf849e-7e46-4e42-a148-05e2c8b8da6e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:07.932041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baf849e-7e46-4e42-a148-05e2c8b8da6e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:07.970950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f526a6e-520a-491c-b7d1-cda3a36776cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:07.971988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f526a6e-520a-491c-b7d1-cda3a36776cf None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:07.972476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f526a6e-520a-491c-b7d1-cda3a36776cf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:07.972969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f526a6e-520a-491c-b7d1-cda3a36776cf None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:07.973884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f526a6e-520a-491c-b7d1-cda3a36776cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:07.975444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f526a6e-520a-491c-b7d1-cda3a36776cf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:07.982186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 466/925] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 12 13:15:07 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:15:08.156899 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbdcb2-137b-4811-a87a-facb6d5870ff None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:08.157190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbdcb2-137b-4811-a87a-facb6d5870ff None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:08.157395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebbdcb2-137b-4811-a87a-facb6d5870ff None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:08.159044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 460/926] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:08.164159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5afeef-b45f-426d-af55-82ef86f35409 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:08.164454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5afeef-b45f-426d-af55-82ef86f35409 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:08.164732 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5afeef-b45f-426d-af55-82ef86f35409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:08.182863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bd5afeef-b45f-426d-af55-82ef86f35409 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:08.184055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bd5afeef-b45f-426d-af55-82ef86f35409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:08.215959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:08.259163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 467/927] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:08.282256 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbdcb2-137b-4811-a87a-facb6d5870ff None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:08.331745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:08.399586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebbdcb2-137b-4811-a87a-facb6d5870ff None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:08.437719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43469e1f-776e-4e81-b183-33876ea91074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:08.438847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469e1f-776e-4e81-b183-33876ea91074 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:08.439316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469e1f-776e-4e81-b183-33876ea91074 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:08.439740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43469e1f-776e-4e81-b183-33876ea91074 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:08.440556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43469e1f-776e-4e81-b183-33876ea91074 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:08.442107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43469e1f-776e-4e81-b183-33876ea91074 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-43469e1f-776e-4e81-b183-33876ea91074 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:08.455567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:08.456718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 12 13:15:08.457804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:08.460380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 461/928] 10.222.0.23 () {56 vars in 1132 bytes} [Tue Jul 12 13:15:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:08.466703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5afeef-b45f-426d-af55-82ef86f35409 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:08.474851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9779df-84de-48e5-8dfb-fe8eb6773d49 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:08.475144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9779df-84de-48e5-8dfb-fe8eb6773d49 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:08.475428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9779df-84de-48e5-8dfb-fe8eb6773d49 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:08.476904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 462/929] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:08.484424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a53cb-8d72-4650-92ce-d83fb0791d71 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:08.484719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a53cb-8d72-4650-92ce-d83fb0791d71 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:08.485000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7a53cb-8d72-4650-92ce-d83fb0791d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:08.502150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ca7a53cb-8d72-4650-92ce-d83fb0791d71 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:08.503452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ca7a53cb-8d72-4650-92ce-d83fb0791d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:08.576459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 463/930] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:08.587751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5afeef-b45f-426d-af55-82ef86f35409 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:08.599331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a53cb-8d72-4650-92ce-d83fb0791d71 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:08.627436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9fd2a-0b44-4dd9-8ba3-87f117ae36ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:08.628109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9fd2a-0b44-4dd9-8ba3-87f117ae36ec None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:08.628665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9fd2a-0b44-4dd9-8ba3-87f117ae36ec None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:08.628948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb9fd2a-0b44-4dd9-8ba3-87f117ae36ec None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:08.630144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9fd2a-0b44-4dd9-8ba3-87f117ae36ec None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:08.630741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9fd2a-0b44-4dd9-8ba3-87f117ae36ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:08.632684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb9fd2a-0b44-4dd9-8ba3-87f117ae36ec None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:08.642719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:08.643880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 468/931] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:08.649353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9fd2a-0b44-4dd9-8ba3-87f117ae36ec None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:08.685760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:08.724997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7a53cb-8d72-4650-92ce-d83fb0791d71 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:08.761063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00622fa9-5d5d-4eb8-b404-0437adb8a148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:08.761300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb9fd2a-0b44-4dd9-8ba3-87f117ae36ec None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:08.761977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00622fa9-5d5d-4eb8-b404-0437adb8a148 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:08.762266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00622fa9-5d5d-4eb8-b404-0437adb8a148 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:08.762540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00622fa9-5d5d-4eb8-b404-0437adb8a148 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:08.775480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00622fa9-5d5d-4eb8-b404-0437adb8a148 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-00622fa9-5d5d-4eb8-b404-0437adb8a148 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:08.785320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:08.786331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:08.787397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 464/932] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 12 13:15:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:08.792967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00622fa9-5d5d-4eb8-b404-0437adb8a148 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:08.807578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8ab1e-a46d-45b3-a3de-16a36b190e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:08.808242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8ab1e-a46d-45b3-a3de-16a36b190e7f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:08.808527 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8ab1e-a46d-45b3-a3de-16a36b190e7f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:08.808810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b8ab1e-a46d-45b3-a3de-16a36b190e7f None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:08.809567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8ab1e-a46d-45b3-a3de-16a36b190e7f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:08.809873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8ab1e-a46d-45b3-a3de-16a36b190e7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:08.812177 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b8ab1e-a46d-45b3-a3de-16a36b190e7f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:08.828469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 469/933] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:08.829338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8ab1e-a46d-45b3-a3de-16a36b190e7f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:08.913832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:08.920427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00622fa9-5d5d-4eb8-b404-0437adb8a148 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:08.958563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084718d2-2db4-419c-955d-4d7b59205214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:08.959475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084718d2-2db4-419c-955d-4d7b59205214 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:08.959834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084718d2-2db4-419c-955d-4d7b59205214 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:08.959977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:08.960120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084718d2-2db4-419c-955d-4d7b59205214 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:08.961851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084718d2-2db4-419c-955d-4d7b59205214 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:08.962165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084718d2-2db4-419c-955d-4d7b59205214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:08.964264 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084718d2-2db4-419c-955d-4d7b59205214 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:08.968984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b8ab1e-a46d-45b3-a3de-16a36b190e7f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:08.974854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 465/934] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 12 13:15:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:08.980679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084718d2-2db4-419c-955d-4d7b59205214 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:09.013901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7ee6-693f-4f8d-99cd-fc98bf5f8799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:09.015181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7ee6-693f-4f8d-99cd-fc98bf5f8799 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:09.015487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7ee6-693f-4f8d-99cd-fc98bf5f8799 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:09.015736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0e7ee6-693f-4f8d-99cd-fc98bf5f8799 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:09.016462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7ee6-693f-4f8d-99cd-fc98bf5f8799 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:09.018048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0e7ee6-693f-4f8d-99cd-fc98bf5f8799 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:09.045117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 470/935] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 12 13:15:08 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:09.097945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084718d2-2db4-419c-955d-4d7b59205214 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:09.142902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dd9d3-a7f2-4912-8d9f-8d5ca1cdcd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:09.143718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd9d3-a7f2-4912-8d9f-8d5ca1cdcd1c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:09.144008 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd9d3-a7f2-4912-8d9f-8d5ca1cdcd1c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:09.144289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0dd9d3-a7f2-4912-8d9f-8d5ca1cdcd1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:09.157080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dd9d3-a7f2-4912-8d9f-8d5ca1cdcd1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:09.158598 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0dd9d3-a7f2-4912-8d9f-8d5ca1cdcd1c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-8d0dd9d3-a7f2-4912-8d9f-8d5ca1cdcd1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:09.178076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:09.181005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:09.182817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:09.182817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:09.182817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:09.182817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:09.182817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 466/936] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:09.188465 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7ee6-693f-4f8d-99cd-fc98bf5f8799 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:09.303323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0e7ee6-693f-4f8d-99cd-fc98bf5f8799 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:09.340002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714673b-ff8e-471c-b794-4381ad181742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:09.340670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714673b-ff8e-471c-b794-4381ad181742 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:09.340951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714673b-ff8e-471c-b794-4381ad181742 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:09.341223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714673b-ff8e-471c-b794-4381ad181742 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:09.342008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714673b-ff8e-471c-b794-4381ad181742 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:09.342312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714673b-ff8e-471c-b794-4381ad181742 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:09.343801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714673b-ff8e-471c-b794-4381ad181742 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:09.363965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 471/937] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:09.369686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dd9d3-a7f2-4912-8d9f-8d5ca1cdcd1c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:09.379744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:09.473823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0dd9d3-a7f2-4912-8d9f-8d5ca1cdcd1c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:09.509178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c1bb5-dcde-47a3-851e-b9b474e1a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:09.509949 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c1bb5-dcde-47a3-851e-b9b474e1a16c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:09.510236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c1bb5-dcde-47a3-851e-b9b474e1a16c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:09.510514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c1bb5-dcde-47a3-851e-b9b474e1a16c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:09.518520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c1bb5-dcde-47a3-851e-b9b474e1a16c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:09.521411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c1bb5-dcde-47a3-851e-b9b474e1a16c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b85c1bb5-dcde-47a3-851e-b9b474e1a16c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:15:09.537820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:09.538859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:15:09.539827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:09.547430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 467/938] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:15:09 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:09.548991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714673b-ff8e-471c-b794-4381ad181742 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:09.565587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:09.657145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714673b-ff8e-471c-b794-4381ad181742 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:09.698877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaea4abb-345c-475a-84af-857bf485a82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:09.699550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea4abb-345c-475a-84af-857bf485a82e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:09.699830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea4abb-345c-475a-84af-857bf485a82e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:09.700102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaea4abb-345c-475a-84af-857bf485a82e None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:09.700944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea4abb-345c-475a-84af-857bf485a82e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:09.701252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea4abb-345c-475a-84af-857bf485a82e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:09.703061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaea4abb-345c-475a-84af-857bf485a82e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:09.721302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 472/939] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 12 13:15:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:09.730420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c1bb5-dcde-47a3-851e-b9b474e1a16c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:09.761475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0352d69-0533-45c8-95e5-6e36f63a26f2 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:09.761679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0352d69-0533-45c8-95e5-6e36f63a26f2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:09.761989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0352d69-0533-45c8-95e5-6e36f63a26f2 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:09.763596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 473/940] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:09.768555 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5bf1d-8649-487f-9be5-1bd570181096 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:09.768843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5bf1d-8649-487f-9be5-1bd570181096 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:09.769123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5bf1d-8649-487f-9be5-1bd570181096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:09.786401 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-67a5bf1d-8649-487f-9be5-1bd570181096 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:09.787607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-67a5bf1d-8649-487f-9be5-1bd570181096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:09.808702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:09.858976 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c1bb5-dcde-47a3-851e-b9b474e1a16c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:09.860981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:09.874163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 474/941] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:09.893086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5bf1d-8649-487f-9be5-1bd570181096 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:09.901329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac6e08-ef8d-4db8-89d1-300cf04aef19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:09.902967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac6e08-ef8d-4db8-89d1-300cf04aef19 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:09.903467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac6e08-ef8d-4db8-89d1-300cf04aef19 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:09.903955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ac6e08-ef8d-4db8-89d1-300cf04aef19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:09.920195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac6e08-ef8d-4db8-89d1-300cf04aef19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:09.922296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ac6e08-ef8d-4db8-89d1-300cf04aef19 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b2ac6e08-ef8d-4db8-89d1-300cf04aef19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:09.941306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:09.942306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:09.943186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:09.943186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:09.943186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:09.943186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:09.943186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 468/942] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:09.948456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac6e08-ef8d-4db8-89d1-300cf04aef19 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:10.024182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5bf1d-8649-487f-9be5-1bd570181096 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:10.062204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:10.063199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.063670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.064128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.064529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ac6e08-ef8d-4db8-89d1-300cf04aef19 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:10.065209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:10.067045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:10.068604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WARNING keystone.server.flask.common [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 12 13:15:10.082411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:10.089268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.sql.core [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 13:15:10.089268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 13:15:10.089268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [parameters: {'id': 'f3b487c8da6e41f495e641ac4f733eb0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 13:15:10.089268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80287) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 13:15:10.102014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a23f1-d301-47cd-be62-fc3d7b6949c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:10.102818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a23f1-d301-47cd-be62-fc3d7b6949c8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.103170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a23f1-d301-47cd-be62-fc3d7b6949c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.103561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3a23f1-d301-47cd-be62-fc3d7b6949c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.104368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a23f1-d301-47cd-be62-fc3d7b6949c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:10.104793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a23f1-d301-47cd-be62-fc3d7b6949c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:10.106325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3a23f1-d301-47cd-be62-fc3d7b6949c8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f3b487c8da6e41f495e641ac4f733eb0.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f3b487c8da6e41f495e641ac4f733eb0. Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:15:10.112825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 12 13:15:10.113667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application yield resource Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise self.value Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application session.commit() Jul 12 13:15:10.114625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.flush() Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application compat.raise_( Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 13:15:10.115497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 13:15:10.116388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:15:10.117204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [parameters: {'id': 'f3b487c8da6e41f495e641ac4f733eb0', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:10.118100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:10.118899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f3b487c8da6e41f495e641ac4f733eb0. Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.121641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 475/943] 10.222.0.23 () {60 vars in 1140 bytes} [Tue Jul 12 13:15:09 2022] POST /identity/v3/projects => generated 262 bytes in 226 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:10.125411 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:10.132806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 469/944] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:10.141517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a23f1-d301-47cd-be62-fc3d7b6949c8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:10.190803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:10.240851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e681e6d9-8bcf-4365-bfa9-9b1ead887857 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:10.256934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3a23f1-d301-47cd-be62-fc3d7b6949c8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:10.284047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45a414-06ae-4f90-9919-5b50f0f3f50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:10.284792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45a414-06ae-4f90-9919-5b50f0f3f50c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.285076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45a414-06ae-4f90-9919-5b50f0f3f50c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.285354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45a414-06ae-4f90-9919-5b50f0f3f50c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.292431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45a414-06ae-4f90-9919-5b50f0f3f50c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:10.294142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c45a414-06ae-4f90-9919-5b50f0f3f50c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:10.295058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3b331f-01a4-4d70-806f-331d6856eae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:10.296597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b331f-01a4-4d70-806f-331d6856eae8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.296891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b331f-01a4-4d70-806f-331d6856eae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.297173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3b331f-01a4-4d70-806f-331d6856eae8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.304944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b331f-01a4-4d70-806f-331d6856eae8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:10.306578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3b331f-01a4-4d70-806f-331d6856eae8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-4c45a414-06ae-4f90-9919-5b50f0f3f50c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:15:10.308784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:10.309768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:10.310640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:15:10.310640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:15:10.310640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:15:10.310640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:15:10.310640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:15:10.310640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:15:10.310640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:10.310640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.310640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 476/945] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:10.311912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25946a5-ead2-4ff8-be58-2c5820256954 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.311912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25946a5-ead2-4ff8-be58-2c5820256954 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.312341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25946a5-ead2-4ff8-be58-2c5820256954 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.316734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 477/946] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:10.320861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25946a5-ead2-4ff8-be58-2c5820256954 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-2f3b331f-01a4-4d70-806f-331d6856eae8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:15:10.328496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:10.329517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:10.330426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:15:10.330426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:15:10.330426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:15:10.330426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:15:10.330426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:15:10.330426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:15:10.330426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:10.330426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:10.334868 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 470/947] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:10.336172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e0e2c-11b2-4b5f-a9d3-6354bac3cec4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.336471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e0e2c-11b2-4b5f-a9d3-6354bac3cec4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.336753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e0e2c-11b2-4b5f-a9d3-6354bac3cec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.354765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a88e0e2c-11b2-4b5f-a9d3-6354bac3cec4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:10.355974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a88e0e2c-11b2-4b5f-a9d3-6354bac3cec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:10.391853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:10.441675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 471/948] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:10.442394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e0e2c-11b2-4b5f-a9d3-6354bac3cec4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:10.450242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25946a5-ead2-4ff8-be58-2c5820256954 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:10.470538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:10.490173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a16b1-edf6-48dd-aba1-bc5b81a1e0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:10.490838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a16b1-edf6-48dd-aba1-bc5b81a1e0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.491137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a16b1-edf6-48dd-aba1-bc5b81a1e0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.491403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a16b1-edf6-48dd-aba1-bc5b81a1e0e4 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.492438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a16b1-edf6-48dd-aba1-bc5b81a1e0e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:10.492438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a16b1-edf6-48dd-aba1-bc5b81a1e0e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:10.493936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a16b1-edf6-48dd-aba1-bc5b81a1e0e4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:10.517471 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 478/949] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:15:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:10.517471 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a16b1-edf6-48dd-aba1-bc5b81a1e0e4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:10.559787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88e0e2c-11b2-4b5f-a9d3-6354bac3cec4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:10.602719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22b0a0-cbfa-465b-8218-758ac3640753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:10.603532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22b0a0-cbfa-465b-8218-758ac3640753 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.603890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22b0a0-cbfa-465b-8218-758ac3640753 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.604230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef22b0a0-cbfa-465b-8218-758ac3640753 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.605105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22b0a0-cbfa-465b-8218-758ac3640753 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:10.605527 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22b0a0-cbfa-465b-8218-758ac3640753 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:10.607182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef22b0a0-cbfa-465b-8218-758ac3640753 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:10.621688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 472/950] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:15:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:10.628262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22b0a0-cbfa-465b-8218-758ac3640753 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:10.645629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a16b1-edf6-48dd-aba1-bc5b81a1e0e4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:10.661840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:10.698886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1573cd-e5a1-4ff1-bae7-1558b01676e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:10.700566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1573cd-e5a1-4ff1-bae7-1558b01676e8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.701179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1573cd-e5a1-4ff1-bae7-1558b01676e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.701485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1573cd-e5a1-4ff1-bae7-1558b01676e8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.702896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1573cd-e5a1-4ff1-bae7-1558b01676e8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:10.704117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1573cd-e5a1-4ff1-bae7-1558b01676e8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-ba1573cd-e5a1-4ff1-bae7-1558b01676e8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:10.719408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:10.720486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:10.721341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 479/951] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:10.730197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1573cd-e5a1-4ff1-bae7-1558b01676e8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:10.752337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef22b0a0-cbfa-465b-8218-758ac3640753 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:10.784686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:10.798021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b31d8d-466d-492c-a78d-7279689cd603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:10.799281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b31d8d-466d-492c-a78d-7279689cd603 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.799568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b31d8d-466d-492c-a78d-7279689cd603 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.799842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b31d8d-466d-492c-a78d-7279689cd603 None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b26fa3bc9c0d4e22a8abc5a4d98c9995/roles/4c2749d472714b3c85b29f3732da518e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.803505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b31d8d-466d-492c-a78d-7279689cd603 None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=b26fa3bc9c0d4e22a8abc5a4d98c9995, role_id=4c2749d472714b3c85b29f3732da518e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:10.804400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b31d8d-466d-492c-a78d-7279689cd603 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:10.820824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 473/952] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:15:10 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/b26fa3bc9c0d4e22a8abc5a4d98c9995/roles/4c2749d472714b3c85b29f3732da518e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:15:10.865593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1573cd-e5a1-4ff1-bae7-1558b01676e8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:10.907943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851dc91-c321-4165-96ab-4bf468a4aa7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:10.909042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851dc91-c321-4165-96ab-4bf468a4aa7d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:10.909570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851dc91-c321-4165-96ab-4bf468a4aa7d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:10.910057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2851dc91-c321-4165-96ab-4bf468a4aa7d None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:10.910994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851dc91-c321-4165-96ab-4bf468a4aa7d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:10.911560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851dc91-c321-4165-96ab-4bf468a4aa7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:10.913293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2851dc91-c321-4165-96ab-4bf468a4aa7d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:10.919817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 480/953] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:10.925729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b31d8d-466d-492c-a78d-7279689cd603 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:11.025828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:11.042418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b31d8d-466d-492c-a78d-7279689cd603 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:11.078193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30eea35-d800-40de-947d-de2fd7f59383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:11.078784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30eea35-d800-40de-947d-de2fd7f59383 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:11.079071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30eea35-d800-40de-947d-de2fd7f59383 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:11.079348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30eea35-d800-40de-947d-de2fd7f59383 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:11.080069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30eea35-d800-40de-947d-de2fd7f59383 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:11.080378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30eea35-d800-40de-947d-de2fd7f59383 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:11.081848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30eea35-d800-40de-947d-de2fd7f59383 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:11.087289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 474/954] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:11.093557 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851dc91-c321-4165-96ab-4bf468a4aa7d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:11.205442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2851dc91-c321-4165-96ab-4bf468a4aa7d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:11.207833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:11.242249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d97f41-39e2-42cc-af46-f8418a3b0a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:11.242920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d97f41-39e2-42cc-af46-f8418a3b0a03 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:11.243205 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d97f41-39e2-42cc-af46-f8418a3b0a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:11.243484 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d97f41-39e2-42cc-af46-f8418a3b0a03 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:11.244230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d97f41-39e2-42cc-af46-f8418a3b0a03 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:11.245626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d97f41-39e2-42cc-af46-f8418a3b0a03 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:11.272062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 481/955] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 12 13:15:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:11.299581 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d8ca-7701-4df6-a167-720c50307a00 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:11.299875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d8ca-7701-4df6-a167-720c50307a00 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:11.300162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c993d8ca-7701-4df6-a167-720c50307a00 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:11.302039 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 475/956] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:11.309133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c109-d47a-466c-aa96-b38b2a0a123c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:11.309585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c109-d47a-466c-aa96-b38b2a0a123c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:11.309994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6c109-d47a-466c-aa96-b38b2a0a123c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:11.332408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0fc6c109-d47a-466c-aa96-b38b2a0a123c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:11.334083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0fc6c109-d47a-466c-aa96-b38b2a0a123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:11.425583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 482/957] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:11.448402 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d8ca-7701-4df6-a167-720c50307a00 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:11.556467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c993d8ca-7701-4df6-a167-720c50307a00 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:11.592374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e18af-234f-467e-a8f0-bf9f26d1ef72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:11.593349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e18af-234f-467e-a8f0-bf9f26d1ef72 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:11.593658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e18af-234f-467e-a8f0-bf9f26d1ef72 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:11.593935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e18af-234f-467e-a8f0-bf9f26d1ef72 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:11.594593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e18af-234f-467e-a8f0-bf9f26d1ef72 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:11.595959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e18af-234f-467e-a8f0-bf9f26d1ef72 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-078e18af-234f-467e-a8f0-bf9f26d1ef72 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:11.608859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:11.610049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:11.611611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 476/958] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:11.616198 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6c109-d47a-466c-aa96-b38b2a0a123c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:11.712366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:11.729714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6c109-d47a-466c-aa96-b38b2a0a123c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:11.767166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4aabd-9d38-4ad8-aaa7-c71126502b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:11.768204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4aabd-9d38-4ad8-aaa7-c71126502b2a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:11.768724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4aabd-9d38-4ad8-aaa7-c71126502b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:11.769208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4aabd-9d38-4ad8-aaa7-c71126502b2a None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:11.770261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4aabd-9d38-4ad8-aaa7-c71126502b2a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:11.770778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4aabd-9d38-4ad8-aaa7-c71126502b2a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:11.772371 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4aabd-9d38-4ad8-aaa7-c71126502b2a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:11.778081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 483/959] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:15:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:11.785855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e18af-234f-467e-a8f0-bf9f26d1ef72 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:11.806962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:11.892131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e18af-234f-467e-a8f0-bf9f26d1ef72 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:11.927333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb9955-a268-4d9d-ae83-c325f57726d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:11.928001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9955-a268-4d9d-ae83-c325f57726d8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:11.928288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9955-a268-4d9d-ae83-c325f57726d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:11.928566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb9955-a268-4d9d-ae83-c325f57726d8 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:11.929286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb9955-a268-4d9d-ae83-c325f57726d8 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:11.929628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb9955-a268-4d9d-ae83-c325f57726d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:11.930994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb9955-a268-4d9d-ae83-c325f57726d8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:11.935975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 477/960] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:15:11 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:12.727632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd855ac-ff8a-4cdc-84ba-a926fd03295f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:12.728147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd855ac-ff8a-4cdc-84ba-a926fd03295f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:12.728225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd855ac-ff8a-4cdc-84ba-a926fd03295f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:12.729848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 484/961] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:12.735616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb9055-5af3-45b7-a1a8-3e96c848fb75 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:12.735910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb9055-5af3-45b7-a1a8-3e96c848fb75 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:12.736187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb9055-5af3-45b7-a1a8-3e96c848fb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:12.753780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-55eb9055-5af3-45b7-a1a8-3e96c848fb75 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:12.754969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-55eb9055-5af3-45b7-a1a8-3e96c848fb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:12.820757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 478/962] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:12.845361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd855ac-ff8a-4cdc-84ba-a926fd03295f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:12.956359 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd855ac-ff8a-4cdc-84ba-a926fd03295f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:12.991542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86f7b94-deff-48ac-aecd-fd7a94232606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:12.992545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f7b94-deff-48ac-aecd-fd7a94232606 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:12.993040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f7b94-deff-48ac-aecd-fd7a94232606 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:12.993570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86f7b94-deff-48ac-aecd-fd7a94232606 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:12.994512 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f7b94-deff-48ac-aecd-fd7a94232606 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:12.996149 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86f7b94-deff-48ac-aecd-fd7a94232606 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:13.011953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.sql.core [None req-e86f7b94-deff-48ac-aecd-fd7a94232606 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 13:15:13.011953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 13:15:13.011953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [parameters: {'user_id': '5df8befee18447e598c6366181f4d75f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 13:15:13.011953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80287) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e86f7b94-deff-48ac-aecd-fd7a94232606 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jul 12 13:15:13.020337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:15:13.021222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application yield resource Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application next(self.gen) Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.rollback() Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.force_reraise() Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise self.value Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application session.commit() Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 12 13:15:13.022814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._prepare_impl() Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.session.flush() Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._flush(objects) Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application compat.raise_( Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application flush_context.execute() Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rec.execute(self) Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 12 13:15:13.023965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 12 13:15:13.025104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result = self._query(query) Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application conn.query(q) Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application result.read() Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 12 13:15:13.027148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [parameters: {'user_id': '5df8befee18447e598c6366181f4d75f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:13.029188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 12 13:15:13.031626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 12 13:15:13.033848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 12 13:15:13.033848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 12 13:15:13.033848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 12 13:15:13.033848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 12 13:15:13.033848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 12 13:15:13.033848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:13.033848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 485/963] 10.222.0.23 () {60 vars in 1131 bytes} [Tue Jul 12 13:15:12 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:13.038386 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb9055-5af3-45b7-a1a8-3e96c848fb75 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:13.052024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:13.053200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153692ee-925e-4b41-aa61-31fd2b93c216 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.053522 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153692ee-925e-4b41-aa61-31fd2b93c216 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.053799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153692ee-925e-4b41-aa61-31fd2b93c216 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.063433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 486/964] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:13 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:13.064927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a5bf5-50ea-47ea-bb9d-34ab98d46c42 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.065214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a5bf5-50ea-47ea-bb9d-34ab98d46c42 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.065523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424a5bf5-50ea-47ea-bb9d-34ab98d46c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.084125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-424a5bf5-50ea-47ea-bb9d-34ab98d46c42 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:13.085693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-424a5bf5-50ea-47ea-bb9d-34ab98d46c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:13.102568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:13.169332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb9055-5af3-45b7-a1a8-3e96c848fb75 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:13.175209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 487/965] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:13.198417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a5bf5-50ea-47ea-bb9d-34ab98d46c42 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:13.211796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de84a5-a3ae-4475-a0d9-224e79637e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:13.213206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de84a5-a3ae-4475-a0d9-224e79637e50 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.213742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de84a5-a3ae-4475-a0d9-224e79637e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.214685 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5de84a5-a3ae-4475-a0d9-224e79637e50 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.227304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de84a5-a3ae-4475-a0d9-224e79637e50 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:13.229102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5de84a5-a3ae-4475-a0d9-224e79637e50 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b5de84a5-a3ae-4475-a0d9-224e79637e50 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:15:13.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:13.251371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 12 13:15:13.252327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:13.259676 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 479/966] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:15:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:13.262076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de84a5-a3ae-4475-a0d9-224e79637e50 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:13.334029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424a5bf5-50ea-47ea-bb9d-34ab98d46c42 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:13.352721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:13.387230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5de84a5-a3ae-4475-a0d9-224e79637e50 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:13.388967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1d182-593f-4752-893c-e2fcc4910d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:13.391289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1d182-593f-4752-893c-e2fcc4910d86 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.391816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1d182-593f-4752-893c-e2fcc4910d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.392301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd1d182-593f-4752-893c-e2fcc4910d86 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.407825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd1d182-593f-4752-893c-e2fcc4910d86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-5fd1d182-593f-4752-893c-e2fcc4910d86 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:13.416361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:13.417361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:13.418828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 488/967] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 12 13:15:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:13.424188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1d182-593f-4752-893c-e2fcc4910d86 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:13.427293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:13.438827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6bc743-c968-44eb-b6bd-eab589633c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:13.440579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bc743-c968-44eb-b6bd-eab589633c68 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.441391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bc743-c968-44eb-b6bd-eab589633c68 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.441885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6bc743-c968-44eb-b6bd-eab589633c68 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.443194 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bc743-c968-44eb-b6bd-eab589633c68 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:13.444404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bc743-c968-44eb-b6bd-eab589633c68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:13.448966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6bc743-c968-44eb-b6bd-eab589633c68 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:13.492702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 480/968] 10.222.0.23 () {56 vars in 1101 bytes} [Tue Jul 12 13:15:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:13.494517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750c2d2-5835-447c-8a33-1c2d4c077198 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.495022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750c2d2-5835-447c-8a33-1c2d4c077198 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.495510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750c2d2-5835-447c-8a33-1c2d4c077198 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.497398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 481/969] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:13.504441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce12444-71ab-4042-8d42-35e23804c455 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.504757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce12444-71ab-4042-8d42-35e23804c455 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.505018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce12444-71ab-4042-8d42-35e23804c455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.524356 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-dce12444-71ab-4042-8d42-35e23804c455 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:13.525731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dce12444-71ab-4042-8d42-35e23804c455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:13.549918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd1d182-593f-4752-893c-e2fcc4910d86 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:13.593898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeab645-11a9-4c55-9000-5fcaa6a4501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:13.594955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeab645-11a9-4c55-9000-5fcaa6a4501f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.595450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeab645-11a9-4c55-9000-5fcaa6a4501f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.595935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeab645-11a9-4c55-9000-5fcaa6a4501f None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.596990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeab645-11a9-4c55-9000-5fcaa6a4501f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:13.597573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeab645-11a9-4c55-9000-5fcaa6a4501f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:13.599325 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeab645-11a9-4c55-9000-5fcaa6a4501f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:13.610627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 489/970] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 12 13:15:13 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:13.615302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 482/971] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:13.616125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeab645-11a9-4c55-9000-5fcaa6a4501f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:13.646141 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce12444-71ab-4042-8d42-35e23804c455 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:13.692117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:13.722463 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:13.735610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeab645-11a9-4c55-9000-5fcaa6a4501f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:13.784566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5a7b5-25bb-4f11-a6ff-7f724ad25a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:13.784566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5a7b5-25bb-4f11-a6ff-7f724ad25a75 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.784566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5a7b5-25bb-4f11-a6ff-7f724ad25a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.784895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5a7b5-25bb-4f11-a6ff-7f724ad25a75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.785351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce12444-71ab-4042-8d42-35e23804c455 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:13.798518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5a7b5-25bb-4f11-a6ff-7f724ad25a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:13.800266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5a7b5-25bb-4f11-a6ff-7f724ad25a75 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e3d5a7b5-25bb-4f11-a6ff-7f724ad25a75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:13.820495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:13.821654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:13.822886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:13.822886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:13.822886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:13.822886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:13.825591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 490/972] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:13.826283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce25679-71dd-49b0-bdbb-d148bfa8db01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:13.827907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce25679-71dd-49b0-bdbb-d148bfa8db01 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.828184 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce25679-71dd-49b0-bdbb-d148bfa8db01 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.828457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce25679-71dd-49b0-bdbb-d148bfa8db01 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.828661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5a7b5-25bb-4f11-a6ff-7f724ad25a75 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:13.829153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce25679-71dd-49b0-bdbb-d148bfa8db01 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:13.830520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce25679-71dd-49b0-bdbb-d148bfa8db01 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-fce25679-71dd-49b0-bdbb-d148bfa8db01 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:13.842774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:13.844057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:13.845103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:13.846703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 483/973] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:13.850302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce25679-71dd-49b0-bdbb-d148bfa8db01 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:13.944473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5a7b5-25bb-4f11-a6ff-7f724ad25a75 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:13.959179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce25679-71dd-49b0-bdbb-d148bfa8db01 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:13.962539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:13.981211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7861e-4b2d-4e24-afb4-b3856227f41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:13.982331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7861e-4b2d-4e24-afb4-b3856227f41e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:13.982620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7861e-4b2d-4e24-afb4-b3856227f41e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:13.983722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7861e-4b2d-4e24-afb4-b3856227f41e None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:13.984465 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7861e-4b2d-4e24-afb4-b3856227f41e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:13.984840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7861e-4b2d-4e24-afb4-b3856227f41e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:13.985072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:13.986259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7861e-4b2d-4e24-afb4-b3856227f41e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:13.998458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 491/974] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:13.999755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc502b11-0580-458b-b7ae-ce695d7f5a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:14.000471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc502b11-0580-458b-b7ae-ce695d7f5a49 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.000788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc502b11-0580-458b-b7ae-ce695d7f5a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.001075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc502b11-0580-458b-b7ae-ce695d7f5a49 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.001824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc502b11-0580-458b-b7ae-ce695d7f5a49 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:14.002136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc502b11-0580-458b-b7ae-ce695d7f5a49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:14.003764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc502b11-0580-458b-b7ae-ce695d7f5a49 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:14.004634 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7861e-4b2d-4e24-afb4-b3856227f41e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:14.014340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 484/975] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:14.016119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc502b11-0580-458b-b7ae-ce695d7f5a49 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:14.103178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7861e-4b2d-4e24-afb4-b3856227f41e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:14.115221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc502b11-0580-458b-b7ae-ce695d7f5a49 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:14.136130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd456c33-0f06-4b2c-b04d-378ecf2cb87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:14.136932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456c33-0f06-4b2c-b04d-378ecf2cb87a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.137223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456c33-0f06-4b2c-b04d-378ecf2cb87a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.137518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd456c33-0f06-4b2c-b04d-378ecf2cb87a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.145063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456c33-0f06-4b2c-b04d-378ecf2cb87a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:14.146532 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd456c33-0f06-4b2c-b04d-378ecf2cb87a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:14.149185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfbcee8-72b2-4eaf-86e6-60c507e21d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:14.149515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbcee8-72b2-4eaf-86e6-60c507e21d81 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.149802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbcee8-72b2-4eaf-86e6-60c507e21d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.150076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfbcee8-72b2-4eaf-86e6-60c507e21d81 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.150785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbcee8-72b2-4eaf-86e6-60c507e21d81 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:14.151092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbcee8-72b2-4eaf-86e6-60c507e21d81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:14.152434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfbcee8-72b2-4eaf-86e6-60c507e21d81 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:14.157187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 485/976] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:14.159515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbf613-7d45-4ca5-8c5a-da6ef231a28a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.159882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbf613-7d45-4ca5-8c5a-da6ef231a28a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.160263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbf613-7d45-4ca5-8c5a-da6ef231a28a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-dd456c33-0f06-4b2c-b04d-378ecf2cb87a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:15:14.161349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:14.162505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 12 13:15:14.163625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:14.164261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 486/977] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:14.164261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbf613-7d45-4ca5-8c5a-da6ef231a28a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:14.164399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 492/978] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:15:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:14.168147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a8d1a-2e30-4920-afc1-c0b3a291fd43 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.168601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a8d1a-2e30-4920-afc1-c0b3a291fd43 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.168884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285a8d1a-2e30-4920-afc1-c0b3a291fd43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.185486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-285a8d1a-2e30-4920-afc1-c0b3a291fd43 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:14.187035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-285a8d1a-2e30-4920-afc1-c0b3a291fd43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:14.216863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:14.219156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:14.256109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 493/979] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:14.256953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a8d1a-2e30-4920-afc1-c0b3a291fd43 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:14.276515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbf613-7d45-4ca5-8c5a-da6ef231a28a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:14.310908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601853f7-161b-4038-8fe8-cd881b4924de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:14.312379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601853f7-161b-4038-8fe8-cd881b4924de None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.312663 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601853f7-161b-4038-8fe8-cd881b4924de None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.312936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601853f7-161b-4038-8fe8-cd881b4924de None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.313773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601853f7-161b-4038-8fe8-cd881b4924de None admin] RBAC: Adding query filter params (interface=admin, service_id=bacb41317bc449868db96e1ee7d1675b, region_id=RegionOne) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:14.314086 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601853f7-161b-4038-8fe8-cd881b4924de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:14.315457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601853f7-161b-4038-8fe8-cd881b4924de None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:14.320382 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 487/980] 10.222.0.23 () {56 vars in 1249 bytes} [Tue Jul 12 13:15:14 2022] GET /identity/v3/endpoints?service_id=bacb41317bc449868db96e1ee7d1675b&interface=admin®ion_id=RegionOne => generated 193 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:14.321129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601853f7-161b-4038-8fe8-cd881b4924de None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:14.363511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285a8d1a-2e30-4920-afc1-c0b3a291fd43 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:14.397889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b43dd0-31d1-439b-b18d-babc770d6cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:14.398561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b43dd0-31d1-439b-b18d-babc770d6cad None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.398847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b43dd0-31d1-439b-b18d-babc770d6cad None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.399128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b43dd0-31d1-439b-b18d-babc770d6cad None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.399966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b43dd0-31d1-439b-b18d-babc770d6cad None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:14.400284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b43dd0-31d1-439b-b18d-babc770d6cad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:14.401761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b43dd0-31d1-439b-b18d-babc770d6cad None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:14.418025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601853f7-161b-4038-8fe8-cd881b4924de None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:14.418563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 494/981] 10.222.0.23 () {56 vars in 1149 bytes} [Tue Jul 12 13:15:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:14.419362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b43dd0-31d1-439b-b18d-babc770d6cad None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:14.435692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:14.445571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:14.458733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4ad80-2756-46e7-b640-7e1732b5461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:14.459403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4ad80-2756-46e7-b640-7e1732b5461c None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.459691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4ad80-2756-46e7-b640-7e1732b5461c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.459972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4ad80-2756-46e7-b640-7e1732b5461c None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.460709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4ad80-2756-46e7-b640-7e1732b5461c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:14.462109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4ad80-2756-46e7-b640-7e1732b5461c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:14.474962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 488/982] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:15:14 2022] POST /identity/v3/services => generated 242 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:14.475754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4ad80-2756-46e7-b640-7e1732b5461c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:14.536237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b43dd0-31d1-439b-b18d-babc770d6cad None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:14.583147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec71ab-1495-4023-b0a9-b24ec574f8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:14.583827 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec71ab-1495-4023-b0a9-b24ec574f8dd None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.584114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec71ab-1495-4023-b0a9-b24ec574f8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.584395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec71ab-1495-4023-b0a9-b24ec574f8dd None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.585087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec71ab-1495-4023-b0a9-b24ec574f8dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:14.586590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec71ab-1495-4023-b0a9-b24ec574f8dd None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:14.593314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4ad80-2756-46e7-b640-7e1732b5461c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:14.594862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 495/983] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 12 13:15:14 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:15:14.641339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5c05b-35cf-4418-8bea-bc90281312e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:14.641967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5c05b-35cf-4418-8bea-bc90281312e1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.642250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5c05b-35cf-4418-8bea-bc90281312e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.642538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d5c05b-35cf-4418-8bea-bc90281312e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.695270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:14.723716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5c05b-35cf-4418-8bea-bc90281312e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:14.725183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d5c05b-35cf-4418-8bea-bc90281312e1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:14.830686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 489/984] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:14 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:14.836402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec71ab-1495-4023-b0a9-b24ec574f8dd None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:14.923297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:14.940753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec71ab-1495-4023-b0a9-b24ec574f8dd None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:14.975543 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102272a-d15d-4785-8ff1-2342b03e1c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:14.977563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102272a-d15d-4785-8ff1-2342b03e1c9f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:14.977563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102272a-d15d-4785-8ff1-2342b03e1c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:14.977563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c102272a-d15d-4785-8ff1-2342b03e1c9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:14.991477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102272a-d15d-4785-8ff1-2342b03e1c9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:14.992936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c102272a-d15d-4785-8ff1-2342b03e1c9f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c102272a-d15d-4785-8ff1-2342b03e1c9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:15.010950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:15.011955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:15.012828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:15.012828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:15.012828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:15.012828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:15.012828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 496/985] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:15.019391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5c05b-35cf-4418-8bea-bc90281312e1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:15.097037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:15.122087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d5c05b-35cf-4418-8bea-bc90281312e1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:15.156856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa825ab7-9eb6-40ef-b311-143d738ad532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:15.157548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa825ab7-9eb6-40ef-b311-143d738ad532 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:15.157836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa825ab7-9eb6-40ef-b311-143d738ad532 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:15.158112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa825ab7-9eb6-40ef-b311-143d738ad532 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:15.158840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa825ab7-9eb6-40ef-b311-143d738ad532 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:15.159147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa825ab7-9eb6-40ef-b311-143d738ad532 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:15.160508 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa825ab7-9eb6-40ef-b311-143d738ad532 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:15.172206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 490/986] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:15.177535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102272a-d15d-4785-8ff1-2342b03e1c9f None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:15.288151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c102272a-d15d-4785-8ff1-2342b03e1c9f None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:15.322453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f02c28-be4a-46ba-b70e-8665213d774d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:15.323248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f02c28-be4a-46ba-b70e-8665213d774d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:15.323533 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f02c28-be4a-46ba-b70e-8665213d774d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:15.323807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f02c28-be4a-46ba-b70e-8665213d774d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:15.326504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:15.332406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f02c28-be4a-46ba-b70e-8665213d774d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:15.333889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f02c28-be4a-46ba-b70e-8665213d774d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-16f02c28-be4a-46ba-b70e-8665213d774d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:15:15.347776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:15.348707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:15.350039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:15:15.350039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:15:15.350039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:15:15.350039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:15:15.350039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:15:15.350039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:15:15.350039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:15.350039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:15.350758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 497/987] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:15.354658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa825ab7-9eb6-40ef-b311-143d738ad532 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:15.459108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa825ab7-9eb6-40ef-b311-143d738ad532 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:15.491962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bf4c36-dc5d-4b9a-8e10-12d7eb1bae4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:15.492625 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf4c36-dc5d-4b9a-8e10-12d7eb1bae4f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:15.492908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf4c36-dc5d-4b9a-8e10-12d7eb1bae4f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:15.493185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bf4c36-dc5d-4b9a-8e10-12d7eb1bae4f None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:15.494005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf4c36-dc5d-4b9a-8e10-12d7eb1bae4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:15.494317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf4c36-dc5d-4b9a-8e10-12d7eb1bae4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:15.495758 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bf4c36-dc5d-4b9a-8e10-12d7eb1bae4f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:15.497321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:15.510073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 491/988] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:15:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:15.515951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f02c28-be4a-46ba-b70e-8665213d774d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:15.626990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f02c28-be4a-46ba-b70e-8665213d774d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:15.661550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddafb28a-91ef-4ded-8f70-28248160ce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:15.662209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafb28a-91ef-4ded-8f70-28248160ce05 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:15.662493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafb28a-91ef-4ded-8f70-28248160ce05 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:15.662773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddafb28a-91ef-4ded-8f70-28248160ce05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:15.663612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafb28a-91ef-4ded-8f70-28248160ce05 None admin] RBAC: Adding query filter params (role.id=4c2749d472714b3c85b29f3732da518e, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=b26fa3bc9c0d4e22a8abc5a4d98c9995) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:15.663928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafb28a-91ef-4ded-8f70-28248160ce05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:15.665353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddafb28a-91ef-4ded-8f70-28248160ce05 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:15.677593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 498/989] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:15 2022] GET /identity/v3/role_assignments?user.id=b26fa3bc9c0d4e22a8abc5a4d98c9995&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=4c2749d472714b3c85b29f3732da518e => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:16.554416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3213bef-53e9-465a-a45d-67e585a6c2e2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:16.554922 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3213bef-53e9-465a-a45d-67e585a6c2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:16.554922 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3213bef-53e9-465a-a45d-67e585a6c2e2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:16.556275 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 492/990] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:16.561670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb7ff7-21a8-4990-8682-8b4028a97693 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:16.562069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb7ff7-21a8-4990-8682-8b4028a97693 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:16.562289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fb7ff7-21a8-4990-8682-8b4028a97693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:16.580926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-01fb7ff7-21a8-4990-8682-8b4028a97693 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:16.582151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-01fb7ff7-21a8-4990-8682-8b4028a97693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:16.639959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:16.653420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 499/991] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:16.675868 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c2436-87a4-417a-ae35-d91c532ebe4a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:16.676168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c2436-87a4-417a-ae35-d91c532ebe4a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:16.676493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c2436-87a4-417a-ae35-d91c532ebe4a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:16.676706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb7ff7-21a8-4990-8682-8b4028a97693 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:16.678025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 493/992] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:16.682882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23bd07-6758-435d-8454-77fbd215a60c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:16.683176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23bd07-6758-435d-8454-77fbd215a60c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:16.683460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e23bd07-6758-435d-8454-77fbd215a60c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:16.701283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1e23bd07-6758-435d-8454-77fbd215a60c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:16.702833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1e23bd07-6758-435d-8454-77fbd215a60c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:16.775088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 494/993] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:16 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:16.780731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fb7ff7-21a8-4990-8682-8b4028a97693 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:16.799872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23bd07-6758-435d-8454-77fbd215a60c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:16.818303 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fcb8f-2863-42b8-90ab-be7fd1ac91e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:16.819132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fcb8f-2863-42b8-90ab-be7fd1ac91e6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:16.819530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fcb8f-2863-42b8-90ab-be7fd1ac91e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:16.819813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164fcb8f-2863-42b8-90ab-be7fd1ac91e6 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:16.820478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fcb8f-2863-42b8-90ab-be7fd1ac91e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:16.821891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164fcb8f-2863-42b8-90ab-be7fd1ac91e6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-164fcb8f-2863-42b8-90ab-be7fd1ac91e6 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:16.835226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:16.836220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:16.837054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 500/994] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:16.842926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fcb8f-2863-42b8-90ab-be7fd1ac91e6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:16.910283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:16.910482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e23bd07-6758-435d-8454-77fbd215a60c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:16.934924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:16.946960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13ac93-3dbf-475d-ad0b-b4faa14f66fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:16.947808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13ac93-3dbf-475d-ad0b-b4faa14f66fc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:16.948049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13ac93-3dbf-475d-ad0b-b4faa14f66fc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:16.948403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f13ac93-3dbf-475d-ad0b-b4faa14f66fc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:16.949092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13ac93-3dbf-475d-ad0b-b4faa14f66fc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:16.950496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f13ac93-3dbf-475d-ad0b-b4faa14f66fc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:16.960781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164fcb8f-2863-42b8-90ab-be7fd1ac91e6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3f13ac93-3dbf-475d-ad0b-b4faa14f66fc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:16.963635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:16.964752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:16.966076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 495/995] 10.222.0.23 () {56 vars in 1126 bytes} [Tue Jul 12 13:15:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:16.970989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13ac93-3dbf-475d-ad0b-b4faa14f66fc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:17.000025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116046f4-be06-4f68-b30f-0d4fdc6a8793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:17.000708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116046f4-be06-4f68-b30f-0d4fdc6a8793 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.000986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116046f4-be06-4f68-b30f-0d4fdc6a8793 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:17.001266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116046f4-be06-4f68-b30f-0d4fdc6a8793 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:17.002005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116046f4-be06-4f68-b30f-0d4fdc6a8793 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:17.002313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116046f4-be06-4f68-b30f-0d4fdc6a8793 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:17.003676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116046f4-be06-4f68-b30f-0d4fdc6a8793 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:17.008848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 501/996] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:15:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:17.014003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116046f4-be06-4f68-b30f-0d4fdc6a8793 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:17.080801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f13ac93-3dbf-475d-ad0b-b4faa14f66fc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:17.114953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd552e2-5429-4a4a-a6e3-59227fa02321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:17.115615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd552e2-5429-4a4a-a6e3-59227fa02321 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.115902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd552e2-5429-4a4a-a6e3-59227fa02321 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:17.116182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd552e2-5429-4a4a-a6e3-59227fa02321 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:17.116892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd552e2-5429-4a4a-a6e3-59227fa02321 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:17.117202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd552e2-5429-4a4a-a6e3-59227fa02321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:17.118594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd552e2-5429-4a4a-a6e3-59227fa02321 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:17.123412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 496/997] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:17.128932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd552e2-5429-4a4a-a6e3-59227fa02321 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:17.130282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116046f4-be06-4f68-b30f-0d4fdc6a8793 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:17.148418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:17.169275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbce7c-67f1-4594-95f6-b4e1d6b356d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:17.169975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbce7c-67f1-4594-95f6-b4e1d6b356d5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.170267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbce7c-67f1-4594-95f6-b4e1d6b356d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:17.170926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bbce7c-67f1-4594-95f6-b4e1d6b356d5 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:17.171334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbce7c-67f1-4594-95f6-b4e1d6b356d5 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:17.171645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbce7c-67f1-4594-95f6-b4e1d6b356d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:17.173127 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bbce7c-67f1-4594-95f6-b4e1d6b356d5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:17.182781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 502/998] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:15:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:17.183722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbce7c-67f1-4594-95f6-b4e1d6b356d5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:17.190510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:17.246409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd552e2-5429-4a4a-a6e3-59227fa02321 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:17.283723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44088b5a-70b1-46d5-921c-91158a2c8ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:17.284401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44088b5a-70b1-46d5-921c-91158a2c8ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.284689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44088b5a-70b1-46d5-921c-91158a2c8ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:17.284968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44088b5a-70b1-46d5-921c-91158a2c8ce4 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:17.286164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44088b5a-70b1-46d5-921c-91158a2c8ce4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:17.286484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44088b5a-70b1-46d5-921c-91158a2c8ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:17.288411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44088b5a-70b1-46d5-921c-91158a2c8ce4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:17.295451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 497/999] 10.222.0.23 () {56 vars in 1122 bytes} [Tue Jul 12 13:15:17 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:17.299583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bbce7c-67f1-4594-95f6-b4e1d6b356d5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:17.300946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44088b5a-70b1-46d5-921c-91158a2c8ce4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:17.341054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b48cd-e7c4-486d-b717-9ed2b4afb42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:17.341675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b48cd-e7c4-486d-b717-9ed2b4afb42c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.341966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b48cd-e7c4-486d-b717-9ed2b4afb42c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:17.342321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9b48cd-e7c4-486d-b717-9ed2b4afb42c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:17.343892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b48cd-e7c4-486d-b717-9ed2b4afb42c None admin] RBAC: Adding query filter params (interface=public, service_id=2ab00fe4e6ec46dea3883401c686b603, region_id=RegionOne) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:17.344214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b48cd-e7c4-486d-b717-9ed2b4afb42c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:17.345667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9b48cd-e7c4-486d-b717-9ed2b4afb42c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:17.356098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 503/1000] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 12 13:15:17 2022] GET /identity/v3/endpoints?service_id=2ab00fe4e6ec46dea3883401c686b603&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:17.356956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b48cd-e7c4-486d-b717-9ed2b4afb42c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:17.407913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44088b5a-70b1-46d5-921c-91158a2c8ce4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:17.409797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:17.424306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:17.445541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aaa21e-4c4f-481a-85f6-346458691cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:17.446211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aaa21e-4c4f-481a-85f6-346458691cb4 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.446495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aaa21e-4c4f-481a-85f6-346458691cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:17.446774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aaa21e-4c4f-481a-85f6-346458691cb4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:17.447509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aaa21e-4c4f-481a-85f6-346458691cb4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:17.448876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aaa21e-4c4f-481a-85f6-346458691cb4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:17.473406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 498/1001] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 12 13:15:17 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:15:17.479466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9b48cd-e7c4-486d-b717-9ed2b4afb42c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:17.526968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29da94f0-3f7a-49fe-a9eb-9b69ae5add53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:17.527646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da94f0-3f7a-49fe-a9eb-9b69ae5add53 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.527935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da94f0-3f7a-49fe-a9eb-9b69ae5add53 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:17.528211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29da94f0-3f7a-49fe-a9eb-9b69ae5add53 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:17.528896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da94f0-3f7a-49fe-a9eb-9b69ae5add53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:17.530466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29da94f0-3f7a-49fe-a9eb-9b69ae5add53 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:17.545432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 504/1002] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 12 13:15:17 2022] GET /identity/v3/services => generated 1727 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:15:17.739833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d956b-25f3-464f-af4d-66467f972bee None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.740346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d956b-25f3-464f-af4d-66467f972bee None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:17.740427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2d956b-25f3-464f-af4d-66467f972bee None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:17.741838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 499/1003] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:17.746977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192c7f4-82b9-460e-8384-4ba38682b8b6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.747425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192c7f4-82b9-460e-8384-4ba38682b8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:17.747574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192c7f4-82b9-460e-8384-4ba38682b8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:17.765162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3192c7f4-82b9-460e-8384-4ba38682b8b6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:17.766532 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3192c7f4-82b9-460e-8384-4ba38682b8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:17.836371 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 505/1004] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:17 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:17.859201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d956b-25f3-464f-af4d-66467f972bee None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:17.963809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2d956b-25f3-464f-af4d-66467f972bee None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:17.998204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180515-26f4-4e9d-a744-1895cdf995ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:17.999686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180515-26f4-4e9d-a744-1895cdf995ff None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:17.999973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180515-26f4-4e9d-a744-1895cdf995ff None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:18.000250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78180515-26f4-4e9d-a744-1895cdf995ff None admin] PATH_INFO: `/v3/services/image` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:18.000913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78180515-26f4-4e9d-a744-1895cdf995ff None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:18.002308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78180515-26f4-4e9d-a744-1895cdf995ff None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-78180515-26f4-4e9d-a744-1895cdf995ff None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:18.016298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:18.017316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:18.018200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 500/1005] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Jul 12 13:15:17 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:18.023486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192c7f4-82b9-460e-8384-4ba38682b8b6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:18.060547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:18.128340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192c7f4-82b9-460e-8384-4ba38682b8b6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:18.162526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2003d3-bc91-417b-9f68-83855773a7a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:18.163197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2003d3-bc91-417b-9f68-83855773a7a5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:18.163486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2003d3-bc91-417b-9f68-83855773a7a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:18.163767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2003d3-bc91-417b-9f68-83855773a7a5 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:18.164484 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2003d3-bc91-417b-9f68-83855773a7a5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:18.164798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2003d3-bc91-417b-9f68-83855773a7a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:18.166277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2003d3-bc91-417b-9f68-83855773a7a5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:18.172089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 506/1006] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:18.177706 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180515-26f4-4e9d-a744-1895cdf995ff None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:18.220273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:18.290299 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78180515-26f4-4e9d-a744-1895cdf995ff None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:18.325320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ac9f6-2522-4743-b662-39b4db8b2ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:18.326013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac9f6-2522-4743-b662-39b4db8b2ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:18.326301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac9f6-2522-4743-b662-39b4db8b2ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:18.326594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ac9f6-2522-4743-b662-39b4db8b2ad7 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:18.328309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac9f6-2522-4743-b662-39b4db8b2ad7 None admin] RBAC: Adding query filter params (type=image) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:18.328627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac9f6-2522-4743-b662-39b4db8b2ad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:18.330019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ac9f6-2522-4743-b662-39b4db8b2ad7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:18.341476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 501/1007] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:19.518731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f35d9-056a-4ab1-a86c-e498eb0c3f1d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:19.519221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f35d9-056a-4ab1-a86c-e498eb0c3f1d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:19.519221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f35d9-056a-4ab1-a86c-e498eb0c3f1d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:19.520714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 507/1008] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:19.527058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51671e-fc08-4a96-96bb-4c22f0f3326e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:19.527473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51671e-fc08-4a96-96bb-4c22f0f3326e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:19.527671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51671e-fc08-4a96-96bb-4c22f0f3326e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:19.546763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7e51671e-fc08-4a96-96bb-4c22f0f3326e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:19.548198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7e51671e-fc08-4a96-96bb-4c22f0f3326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:19.597550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ab5c5-5ca3-4071-a422-7b4a13cb41d2 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:19.597833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ab5c5-5ca3-4071-a422-7b4a13cb41d2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:19.598109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4ab5c5-5ca3-4071-a422-7b4a13cb41d2 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:19.599723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 508/1009] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:19.604428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4872-3dce-4318-a08a-b7ebb19b3c4f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:19.604716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4872-3dce-4318-a08a-b7ebb19b3c4f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:19.604994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff4872-3dce-4318-a08a-b7ebb19b3c4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:19.620115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 502/1010] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:19.621870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1cff4872-3dce-4318-a08a-b7ebb19b3c4f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:19.623065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1cff4872-3dce-4318-a08a-b7ebb19b3c4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:19.643723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51671e-fc08-4a96-96bb-4c22f0f3326e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:19.677118 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:19.695838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 509/1011] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:19 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:19.719152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff4872-3dce-4318-a08a-b7ebb19b3c4f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:19.745788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51671e-fc08-4a96-96bb-4c22f0f3326e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:19.748251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:19.782191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fc733-bcdf-4c2e-b7cd-8462f85612c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:19.782904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fc733-bcdf-4c2e-b7cd-8462f85612c2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:19.783188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fc733-bcdf-4c2e-b7cd-8462f85612c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:19.783467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335fc733-bcdf-4c2e-b7cd-8462f85612c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:19.784636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fc733-bcdf-4c2e-b7cd-8462f85612c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:19.786097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335fc733-bcdf-4c2e-b7cd-8462f85612c2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:19.787867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 503/1012] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:19.793458 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fc733-bcdf-4c2e-b7cd-8462f85612c2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:19.822245 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff4872-3dce-4318-a08a-b7ebb19b3c4f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:19.855784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55b049-1f65-4e1c-ac71-ffa88375e3ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:19.856578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55b049-1f65-4e1c-ac71-ffa88375e3ce None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:19.856861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55b049-1f65-4e1c-ac71-ffa88375e3ce None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:19.857141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55b049-1f65-4e1c-ac71-ffa88375e3ce None admin] PATH_INFO: `/v3/services/compute` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:19.858385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55b049-1f65-4e1c-ac71-ffa88375e3ce None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:19.859765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da55b049-1f65-4e1c-ac71-ffa88375e3ce None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-da55b049-1f65-4e1c-ac71-ffa88375e3ce None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:19.871298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:19.872272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 12 13:15:19.873106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:19.877682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 510/1013] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:19.878968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55b049-1f65-4e1c-ac71-ffa88375e3ce None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:19.903913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335fc733-bcdf-4c2e-b7cd-8462f85612c2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:19.905686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:19.944254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9498a2-8d1c-4410-950a-f7754a3c47eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:19.944916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9498a2-8d1c-4410-950a-f7754a3c47eb None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:19.945201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9498a2-8d1c-4410-950a-f7754a3c47eb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:19.945507 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9498a2-8d1c-4410-950a-f7754a3c47eb None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:19.946212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9498a2-8d1c-4410-950a-f7754a3c47eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:19.947634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9498a2-8d1c-4410-950a-f7754a3c47eb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:19.978620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:19.989411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 504/1014] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:15:19 2022] POST /identity/v3/projects => generated 311 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:15:19.990570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da55b049-1f65-4e1c-ac71-ffa88375e3ce None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:20.032802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a4b6-a70c-4714-a6f5-fc63a07964a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:20.033455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a4b6-a70c-4714-a6f5-fc63a07964a5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:20.033736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a4b6-a70c-4714-a6f5-fc63a07964a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:20.034020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e91a4b6-a70c-4714-a6f5-fc63a07964a5 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:20.034759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91a4b6-a70c-4714-a6f5-fc63a07964a5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:20.035071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91a4b6-a70c-4714-a6f5-fc63a07964a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:20.036518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e91a4b6-a70c-4714-a6f5-fc63a07964a5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:20.042370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 511/1015] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:15:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:20.048944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9498a2-8d1c-4410-950a-f7754a3c47eb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:20.158122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9498a2-8d1c-4410-950a-f7754a3c47eb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:20.192941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0806cd60-e130-4f00-a6ab-92d32823767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:20.193928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806cd60-e130-4f00-a6ab-92d32823767e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:20.194429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806cd60-e130-4f00-a6ab-92d32823767e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:20.194708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0806cd60-e130-4f00-a6ab-92d32823767e None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:20.195428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806cd60-e130-4f00-a6ab-92d32823767e None admin] RBAC: Adding query filter params (type=compute) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:20.195735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806cd60-e130-4f00-a6ab-92d32823767e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:20.197095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0806cd60-e130-4f00-a6ab-92d32823767e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:20.202580 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 505/1016] 10.222.0.23 () {56 vars in 1112 bytes} [Tue Jul 12 13:15:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:20.208486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a4b6-a70c-4714-a6f5-fc63a07964a5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:20.316100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e91a4b6-a70c-4714-a6f5-fc63a07964a5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:20.351117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab1e4e-ac71-4141-9472-594a32d0a73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:20.351789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1e4e-ac71-4141-9472-594a32d0a73e None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:20.352079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1e4e-ac71-4141-9472-594a32d0a73e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:20.352364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab1e4e-ac71-4141-9472-594a32d0a73e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:20.353123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab1e4e-ac71-4141-9472-594a32d0a73e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:20.354671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dab1e4e-ac71-4141-9472-594a32d0a73e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:20.359647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25ee95-8fcb-478f-a3e0-70aadb8fe694 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:20.359939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25ee95-8fcb-478f-a3e0-70aadb8fe694 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:20.360300 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25ee95-8fcb-478f-a3e0-70aadb8fe694 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:20.361743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 506/1017] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:20.366739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ead8d-22a5-4448-bb01-bdb311c86707 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:20.366967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ead8d-22a5-4448-bb01-bdb311c86707 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:20.367232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38ead8d-22a5-4448-bb01-bdb311c86707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:20.379000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 512/1018] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 12 13:15:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:15:20.393427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c38ead8d-22a5-4448-bb01-bdb311c86707 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:20.394779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c38ead8d-22a5-4448-bb01-bdb311c86707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:20.411246 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:20.481429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 507/1019] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:20 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:20.505450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab1e4e-ac71-4141-9472-594a32d0a73e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:20.524762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:20.614247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab1e4e-ac71-4141-9472-594a32d0a73e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:20.649647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7480a72-2c52-4810-a23b-8d6ffe73d740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:20.650329 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7480a72-2c52-4810-a23b-8d6ffe73d740 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:20.650618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7480a72-2c52-4810-a23b-8d6ffe73d740 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:20.650900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7480a72-2c52-4810-a23b-8d6ffe73d740 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:20.651624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7480a72-2c52-4810-a23b-8d6ffe73d740 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:20.653029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7480a72-2c52-4810-a23b-8d6ffe73d740 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:20.669274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 513/1020] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:15:20 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:22.101912 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d341b1-9ffa-4e1a-aea9-173f4cff1052 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.102467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d341b1-9ffa-4e1a-aea9-173f4cff1052 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.102467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d341b1-9ffa-4e1a-aea9-173f4cff1052 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.103971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 508/1021] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:22.109897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ec0f3-3e7d-49fd-be85-18a4d3cd4ff4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.110316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ec0f3-3e7d-49fd-be85-18a4d3cd4ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.110481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ec0f3-3e7d-49fd-be85-18a4d3cd4ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.127218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bb8ec0f3-3e7d-49fd-be85-18a4d3cd4ff4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:22.128416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bb8ec0f3-3e7d-49fd-be85-18a4d3cd4ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:22.196180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:22.209420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 514/1022] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:22.229085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d341b1-9ffa-4e1a-aea9-173f4cff1052 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:22.339304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d341b1-9ffa-4e1a-aea9-173f4cff1052 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:22.373716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4d379-e1a3-43f7-b7c6-da60d70f902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:22.374441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4d379-e1a3-43f7-b7c6-da60d70f902f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.374728 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4d379-e1a3-43f7-b7c6-da60d70f902f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.375007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4d379-e1a3-43f7-b7c6-da60d70f902f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.376187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4d379-e1a3-43f7-b7c6-da60d70f902f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:22.377654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4d379-e1a3-43f7-b7c6-da60d70f902f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:22.383706 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 509/1023] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:22.384934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ec0f3-3e7d-49fd-be85-18a4d3cd4ff4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:22.485041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ec0f3-3e7d-49fd-be85-18a4d3cd4ff4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:22.519880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9a8d0b-4c49-41fd-9a0f-c3d81ef3fc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:22.520862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a8d0b-4c49-41fd-9a0f-c3d81ef3fc2d None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.521330 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a8d0b-4c49-41fd-9a0f-c3d81ef3fc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.521847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9a8d0b-4c49-41fd-9a0f-c3d81ef3fc2d None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.522746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a8d0b-4c49-41fd-9a0f-c3d81ef3fc2d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:22.524363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9a8d0b-4c49-41fd-9a0f-c3d81ef3fc2d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:22.553403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 515/1024] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:15:22 2022] POST /identity/v3/users => generated 285 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:22.624926 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da5152-918c-4a42-a316-db52d1cbf852 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.625147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da5152-918c-4a42-a316-db52d1cbf852 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.625466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22da5152-918c-4a42-a316-db52d1cbf852 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.626896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 510/1025] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:22.632571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5e891-3ec2-407b-bb7d-dd71a9e5c99a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.632859 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5e891-3ec2-407b-bb7d-dd71a9e5c99a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.633141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d5e891-3ec2-407b-bb7d-dd71a9e5c99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.651336 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-52d5e891-3ec2-407b-bb7d-dd71a9e5c99a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:22.652567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-52d5e891-3ec2-407b-bb7d-dd71a9e5c99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:22.708109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:22.708316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba71f99-cfe8-4832-8191-0dc2b2778d97 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.708543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba71f99-cfe8-4832-8191-0dc2b2778d97 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.708733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba71f99-cfe8-4832-8191-0dc2b2778d97 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.710118 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 511/1026] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:22.716127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9904f-6e32-440f-8ae2-3103f5703674 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.716399 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9904f-6e32-440f-8ae2-3103f5703674 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.716681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f9904f-6e32-440f-8ae2-3103f5703674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.730586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 516/1027] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:22.737287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-79f9904f-6e32-440f-8ae2-3103f5703674 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:22.738524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-79f9904f-6e32-440f-8ae2-3103f5703674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:22.746555 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:22.755073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5e891-3ec2-407b-bb7d-dd71a9e5c99a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:22.820055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 512/1028] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:22.844852 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9904f-6e32-440f-8ae2-3103f5703674 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:22.863924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d5e891-3ec2-407b-bb7d-dd71a9e5c99a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:22.903341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d0a9a-8c60-40ef-bd73-1f7782769b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:22.904124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d0a9a-8c60-40ef-bd73-1f7782769b39 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.904411 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d0a9a-8c60-40ef-bd73-1f7782769b39 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.905433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306d0a9a-8c60-40ef-bd73-1f7782769b39 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.917120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306d0a9a-8c60-40ef-bd73-1f7782769b39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-306d0a9a-8c60-40ef-bd73-1f7782769b39 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:22.926163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:22.927109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:22.927936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 517/1029] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 12 13:15:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:22.933058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d0a9a-8c60-40ef-bd73-1f7782769b39 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:22.959910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f9904f-6e32-440f-8ae2-3103f5703674 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:22.985148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:22.995632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d88677-3930-4204-9004-10d339c17939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:22.996432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d88677-3930-4204-9004-10d339c17939 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:22.996721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d88677-3930-4204-9004-10d339c17939 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:22.997007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d88677-3930-4204-9004-10d339c17939 None admin] PATH_INFO: `/v3/services/image` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:22.997779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d88677-3930-4204-9004-10d339c17939 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.000058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d88677-3930-4204-9004-10d339c17939 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-b6d88677-3930-4204-9004-10d339c17939 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:23.011871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:23.012864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:23.013757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 513/1030] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Jul 12 13:15:22 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:23.019154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d88677-3930-4204-9004-10d339c17939 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:23.028336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:23.050051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306d0a9a-8c60-40ef-bd73-1f7782769b39 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.089638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767108b-50c3-4aab-9b71-271a345f2f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.090313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767108b-50c3-4aab-9b71-271a345f2f13 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.090709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767108b-50c3-4aab-9b71-271a345f2f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.090995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b767108b-50c3-4aab-9b71-271a345f2f13 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:23.091890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767108b-50c3-4aab-9b71-271a345f2f13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:23.092233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767108b-50c3-4aab-9b71-271a345f2f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.094369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b767108b-50c3-4aab-9b71-271a345f2f13 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.108789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 518/1031] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 12 13:15:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:23.109867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767108b-50c3-4aab-9b71-271a345f2f13 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:23.130982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d88677-3930-4204-9004-10d339c17939 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.165484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431041-71d2-40b3-afbf-873cd089eb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.166074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431041-71d2-40b3-afbf-873cd089eb9e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.166369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431041-71d2-40b3-afbf-873cd089eb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.166643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36431041-71d2-40b3-afbf-873cd089eb9e None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:23.167369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36431041-71d2-40b3-afbf-873cd089eb9e None admin] RBAC: Adding query filter params (name=image) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:23.167677 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36431041-71d2-40b3-afbf-873cd089eb9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.169041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36431041-71d2-40b3-afbf-873cd089eb9e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.179064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 514/1032] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:23.180324 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431041-71d2-40b3-afbf-873cd089eb9e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:23.222147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b767108b-50c3-4aab-9b71-271a345f2f13 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.235010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:23.261933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27ced7-9b6a-4137-8aa1-555ad03da91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.262784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27ced7-9b6a-4137-8aa1-555ad03da91d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.262916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:23.263088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27ced7-9b6a-4137-8aa1-555ad03da91d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.263343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc27ced7-9b6a-4137-8aa1-555ad03da91d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:23.275100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27ced7-9b6a-4137-8aa1-555ad03da91d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.276541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc27ced7-9b6a-4137-8aa1-555ad03da91d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-dc27ced7-9b6a-4137-8aa1-555ad03da91d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:23.295003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:23.296581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:23.297976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:23.297976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:23.297976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:23.297976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:23.298214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36431041-71d2-40b3-afbf-873cd089eb9e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.301587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 519/1033] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:23.302463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27ced7-9b6a-4137-8aa1-555ad03da91d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:23.333704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9d277-514f-422c-9f96-c84543cbf272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.334457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9d277-514f-422c-9f96-c84543cbf272 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.334655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9d277-514f-422c-9f96-c84543cbf272 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.334930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc9d277-514f-422c-9f96-c84543cbf272 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:23.335772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9d277-514f-422c-9f96-c84543cbf272 None admin] RBAC: Adding query filter params (type=image) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:23.335961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9d277-514f-422c-9f96-c84543cbf272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.337306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc9d277-514f-422c-9f96-c84543cbf272 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.343606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 515/1034] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:23.349196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9d277-514f-422c-9f96-c84543cbf272 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:23.415446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc27ced7-9b6a-4137-8aa1-555ad03da91d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.454494 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5f385-7c27-478c-96a7-ccf7d2188a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.455081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5f385-7c27-478c-96a7-ccf7d2188a1c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.455362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5f385-7c27-478c-96a7-ccf7d2188a1c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.455639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5f385-7c27-478c-96a7-ccf7d2188a1c None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:23.456359 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5f385-7c27-478c-96a7-ccf7d2188a1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:23.456664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5f385-7c27-478c-96a7-ccf7d2188a1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.457980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc9d277-514f-422c-9f96-c84543cbf272 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.458086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5f385-7c27-478c-96a7-ccf7d2188a1c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.472125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 520/1035] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:23.477430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5f385-7c27-478c-96a7-ccf7d2188a1c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:23.494948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea700ec-72b1-404a-ba8f-d10e5c84f846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.495917 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea700ec-72b1-404a-ba8f-d10e5c84f846 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.496427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea700ec-72b1-404a-ba8f-d10e5c84f846 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.496801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea700ec-72b1-404a-ba8f-d10e5c84f846 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:23.497629 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea700ec-72b1-404a-ba8f-d10e5c84f846 None admin] RBAC: Adding query filter params (interface=public, service_id=c66e53cf1acc4f99979b9e17669d62c7, region_id=RegionOne) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:23.497944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea700ec-72b1-404a-ba8f-d10e5c84f846 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.499489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea700ec-72b1-404a-ba8f-d10e5c84f846 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.499946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:23.501711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:23.507421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 516/1036] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 12 13:15:23 2022] GET /identity/v3/endpoints?service_id=c66e53cf1acc4f99979b9e17669d62c7&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:23.512557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea700ec-72b1-404a-ba8f-d10e5c84f846 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:23.586242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5f385-7c27-478c-96a7-ccf7d2188a1c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.612506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea700ec-72b1-404a-ba8f-d10e5c84f846 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.622966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76331744-e1ac-48fa-9319-ca747bef7d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.623794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76331744-e1ac-48fa-9319-ca747bef7d8c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.624079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76331744-e1ac-48fa-9319-ca747bef7d8c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.624350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76331744-e1ac-48fa-9319-ca747bef7d8c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:23.631883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76331744-e1ac-48fa-9319-ca747bef7d8c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.633312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76331744-e1ac-48fa-9319-ca747bef7d8c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.645640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68082d9-3b67-471b-a4ca-238847cbc4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.646309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68082d9-3b67-471b-a4ca-238847cbc4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.646595 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68082d9-3b67-471b-a4ca-238847cbc4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.646877 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68082d9-3b67-471b-a4ca-238847cbc4c1 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:23.647563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68082d9-3b67-471b-a4ca-238847cbc4c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.648911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68082d9-3b67-471b-a4ca-238847cbc4c1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-76331744-e1ac-48fa-9319-ca747bef7d8c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:15:23.649354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:23.650284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:23.651238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 521/1037] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 12 13:15:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:23.656734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76331744-e1ac-48fa-9319-ca747bef7d8c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:23.658969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 517/1038] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 12 13:15:23 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:15:23.757467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:23.780759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76331744-e1ac-48fa-9319-ca747bef7d8c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.818772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23042b75-d123-4cfa-8944-da94310f0c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.819447 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23042b75-d123-4cfa-8944-da94310f0c07 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.819729 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23042b75-d123-4cfa-8944-da94310f0c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.820004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23042b75-d123-4cfa-8944-da94310f0c07 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:23.820843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23042b75-d123-4cfa-8944-da94310f0c07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:23.821148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23042b75-d123-4cfa-8944-da94310f0c07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:23.822608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23042b75-d123-4cfa-8944-da94310f0c07 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:23.840362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 522/1039] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 12 13:15:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:23.846260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68082d9-3b67-471b-a4ca-238847cbc4c1 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:23.913481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:23.955158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68082d9-3b67-471b-a4ca-238847cbc4c1 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:23.989389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3210c-24bc-4372-9804-bcd7701989a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:23.990273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3210c-24bc-4372-9804-bcd7701989a5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:23.990738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3210c-24bc-4372-9804-bcd7701989a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:23.991166 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3210c-24bc-4372-9804-bcd7701989a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:24.074973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3210c-24bc-4372-9804-bcd7701989a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:24.076745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3210c-24bc-4372-9804-bcd7701989a5 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:24.138058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:24.184400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 518/1040] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:23 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:24.186163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23042b75-d123-4cfa-8944-da94310f0c07 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:24.293692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23042b75-d123-4cfa-8944-da94310f0c07 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:24.329441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569b4cd-b2a9-426b-9a2c-c943e990c1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:24.330159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569b4cd-b2a9-426b-9a2c-c943e990c1dc None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:24.330447 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569b4cd-b2a9-426b-9a2c-c943e990c1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:24.330725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c569b4cd-b2a9-426b-9a2c-c943e990c1dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:24.338128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:24.342668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569b4cd-b2a9-426b-9a2c-c943e990c1dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:24.345568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c569b4cd-b2a9-426b-9a2c-c943e990c1dc None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c569b4cd-b2a9-426b-9a2c-c943e990c1dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:24.367535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:24.368487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:24.369355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:24.369355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:24.369355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:24.369355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:24.373452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 523/1041] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:24.374880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3210c-24bc-4372-9804-bcd7701989a5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:24.480221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3210c-24bc-4372-9804-bcd7701989a5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:24.517128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108488c-78b1-4ea6-8796-075ab692e86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:24.517828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108488c-78b1-4ea6-8796-075ab692e86b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:24.518117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108488c-78b1-4ea6-8796-075ab692e86b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:24.518398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108488c-78b1-4ea6-8796-075ab692e86b None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:24.519131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108488c-78b1-4ea6-8796-075ab692e86b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:24.519442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108488c-78b1-4ea6-8796-075ab692e86b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:24.520846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108488c-78b1-4ea6-8796-075ab692e86b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:24.537361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 519/1042] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:24.539045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569b4cd-b2a9-426b-9a2c-c943e990c1dc None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:24.643027 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c569b4cd-b2a9-426b-9a2c-c943e990c1dc None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:24.650060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef476e24-68aa-4d59-ab69-e6fc9b056b0f None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:24.650359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef476e24-68aa-4d59-ab69-e6fc9b056b0f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:24.650634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef476e24-68aa-4d59-ab69-e6fc9b056b0f None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:24.652123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 520/1043] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:24.656905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838845d-4a7e-45ef-8c09-35ba703b7395 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:24.657196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838845d-4a7e-45ef-8c09-35ba703b7395 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:24.657524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6838845d-4a7e-45ef-8c09-35ba703b7395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:24.675096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6838845d-4a7e-45ef-8c09-35ba703b7395 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:24.676353 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6838845d-4a7e-45ef-8c09-35ba703b7395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:24.681496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944a98e-c6cd-4b71-bf40-7539dd61c26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:24.682293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944a98e-c6cd-4b71-bf40-7539dd61c26b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:24.682575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944a98e-c6cd-4b71-bf40-7539dd61c26b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:24.682850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b944a98e-c6cd-4b71-bf40-7539dd61c26b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:24.690752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944a98e-c6cd-4b71-bf40-7539dd61c26b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:24.692183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b944a98e-c6cd-4b71-bf40-7539dd61c26b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:24.694242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-b944a98e-c6cd-4b71-bf40-7539dd61c26b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:15:24.707538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:24.708457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:24.709351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:15:24.709351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:15:24.709351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:15:24.709351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:15:24.709351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:15:24.709351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:15:24.709351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:24.709351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:24.709351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 524/1044] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:24.714266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944a98e-c6cd-4b71-bf40-7539dd61c26b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:24.754466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 521/1045] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:24.767569 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:24.778057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838845d-4a7e-45ef-8c09-35ba703b7395 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:24.825697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b944a98e-c6cd-4b71-bf40-7539dd61c26b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:24.861665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fc57a-9100-4d17-87fa-0d784fbb639b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:24.862566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fc57a-9100-4d17-87fa-0d784fbb639b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:24.863011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fc57a-9100-4d17-87fa-0d784fbb639b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:24.863286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563fc57a-9100-4d17-87fa-0d784fbb639b None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:24.864082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fc57a-9100-4d17-87fa-0d784fbb639b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:24.864398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fc57a-9100-4d17-87fa-0d784fbb639b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:24.865897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563fc57a-9100-4d17-87fa-0d784fbb639b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:24.883754 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 525/1046] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:15:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:24.884756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fc57a-9100-4d17-87fa-0d784fbb639b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:24.890753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6838845d-4a7e-45ef-8c09-35ba703b7395 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:24.927617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13d5f9-c846-4e5b-8ef4-355840f2f8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:24.928337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13d5f9-c846-4e5b-8ef4-355840f2f8bf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:24.928622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13d5f9-c846-4e5b-8ef4-355840f2f8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:24.928895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13d5f9-c846-4e5b-8ef4-355840f2f8bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:24.940526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a13d5f9-c846-4e5b-8ef4-355840f2f8bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-1a13d5f9-c846-4e5b-8ef4-355840f2f8bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:24.948856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:24.949871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:24.950754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 522/1047] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:15:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:24.957014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13d5f9-c846-4e5b-8ef4-355840f2f8bf None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:24.971799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:25.005323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563fc57a-9100-4d17-87fa-0d784fbb639b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:25.015862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:25.040891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c801333-ca54-423b-884f-fe7345ab4404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:25.041844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c801333-ca54-423b-884f-fe7345ab4404 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:25.042126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c801333-ca54-423b-884f-fe7345ab4404 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:25.042403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c801333-ca54-423b-884f-fe7345ab4404 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:25.043218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c801333-ca54-423b-884f-fe7345ab4404 None admin] RBAC: Adding query filter params (role.id=4c2749d472714b3c85b29f3732da518e, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=1cc0050f14664630a36814ca670f124f) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:25.043526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c801333-ca54-423b-884f-fe7345ab4404 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:25.045084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c801333-ca54-423b-884f-fe7345ab4404 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:25.061414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 526/1048] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:24 2022] GET /identity/v3/role_assignments?user.id=1cc0050f14664630a36814ca670f124f&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=4c2749d472714b3c85b29f3732da518e => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:25.076389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13d5f9-c846-4e5b-8ef4-355840f2f8bf None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:25.115237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b8905-5752-4f66-9a67-5a3376c3031d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:25.116007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8905-5752-4f66-9a67-5a3376c3031d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:25.116298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8905-5752-4f66-9a67-5a3376c3031d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:25.116574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b8905-5752-4f66-9a67-5a3376c3031d None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:25.117477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b8905-5752-4f66-9a67-5a3376c3031d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:25.117797 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b8905-5752-4f66-9a67-5a3376c3031d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:25.119279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b8905-5752-4f66-9a67-5a3376c3031d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:25.133490 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 523/1049] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:15:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:25.135616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c801333-ca54-423b-884f-fe7345ab4404 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:25.245660 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c801333-ca54-423b-884f-fe7345ab4404 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:25.280703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd4d39-eb5d-4e51-bc53-43a295dfac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:25.281683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd4d39-eb5d-4e51-bc53-43a295dfac6c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:25.281969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd4d39-eb5d-4e51-bc53-43a295dfac6c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:25.282244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fd4d39-eb5d-4e51-bc53-43a295dfac6c None admin] PATH_INFO: `/v3/users/81829245596a48f4ad5a323ffe74478f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:25.296126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd4d39-eb5d-4e51-bc53-43a295dfac6c None admin] RBAC: Authorizing `identity:get_user(user_id=81829245596a48f4ad5a323ffe74478f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:25.297610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fd4d39-eb5d-4e51-bc53-43a295dfac6c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:25.303851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 527/1050] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:25 2022] GET /identity/v3/users/81829245596a48f4ad5a323ffe74478f => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:25.305094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b8905-5752-4f66-9a67-5a3376c3031d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:25.383560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:25.411479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b8905-5752-4f66-9a67-5a3376c3031d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:25.448437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e8d4ac-d092-4a7e-93ff-ee990cdf9313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:25.449094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d4ac-d092-4a7e-93ff-ee990cdf9313 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:25.449404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d4ac-d092-4a7e-93ff-ee990cdf9313 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:25.449688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e8d4ac-d092-4a7e-93ff-ee990cdf9313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:25.519028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8d4ac-d092-4a7e-93ff-ee990cdf9313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:25.520391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e8d4ac-d092-4a7e-93ff-ee990cdf9313 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:25.593590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:25.625562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 524/1051] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:25 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:25.627199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd4d39-eb5d-4e51-bc53-43a295dfac6c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:25.659672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:25.731627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fd4d39-eb5d-4e51-bc53-43a295dfac6c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:25.740189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aa615-6be8-4e8c-9061-540f3d409944 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:25.740483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aa615-6be8-4e8c-9061-540f3d409944 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:25.740762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401aa615-6be8-4e8c-9061-540f3d409944 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:25.742297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 525/1052] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:25.746896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b4dd3-af6a-427a-a904-8f32f2ef2e50 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:25.747182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b4dd3-af6a-427a-a904-8f32f2ef2e50 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:25.747465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b4dd3-af6a-427a-a904-8f32f2ef2e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:25.765226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4e5b4dd3-af6a-427a-a904-8f32f2ef2e50 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:25.766041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4e5b4dd3-af6a-427a-a904-8f32f2ef2e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:25.768271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf14a46-63c8-4485-ad52-ab301454f0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:25.769217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf14a46-63c8-4485-ad52-ab301454f0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:25.769453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf14a46-63c8-4485-ad52-ab301454f0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:25.769740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf14a46-63c8-4485-ad52-ab301454f0e8 None admin] PATH_INFO: `/v3/projects/0743d4c307104b9eac562dd17591266f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:25.770917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf14a46-63c8-4485-ad52-ab301454f0e8 None admin] RBAC: Authorizing `identity:get_project(project_id=0743d4c307104b9eac562dd17591266f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:25.772342 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf14a46-63c8-4485-ad52-ab301454f0e8 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:25.774083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 528/1053] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:15:25 2022] GET /identity/v3/projects/0743d4c307104b9eac562dd17591266f => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:25.780045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf14a46-63c8-4485-ad52-ab301454f0e8 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:25.840559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 526/1054] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:25 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:25.864262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b4dd3-af6a-427a-a904-8f32f2ef2e50 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:25.879358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:25.884957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf14a46-63c8-4485-ad52-ab301454f0e8 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:25.922201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685d37-df0a-4f5b-b73b-31723929928c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:25.922860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685d37-df0a-4f5b-b73b-31723929928c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:25.923251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685d37-df0a-4f5b-b73b-31723929928c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:25.923426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0685d37-df0a-4f5b-b73b-31723929928c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:25.924253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0685d37-df0a-4f5b-b73b-31723929928c None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=0743d4c307104b9eac562dd17591266f, user.id=81829245596a48f4ad5a323ffe74478f) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:25.924675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0685d37-df0a-4f5b-b73b-31723929928c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:25.926055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0685d37-df0a-4f5b-b73b-31723929928c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:25.941403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 529/1055] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:25 2022] GET /identity/v3/role_assignments?user.id=81829245596a48f4ad5a323ffe74478f&scope.project.id=0743d4c307104b9eac562dd17591266f&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:25.982005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5b4dd3-af6a-427a-a904-8f32f2ef2e50 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:25.984727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:26.022336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a49cf-964a-4d23-a993-0eacdf1e5e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:26.023064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a49cf-964a-4d23-a993-0eacdf1e5e10 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:26.023347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a49cf-964a-4d23-a993-0eacdf1e5e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:26.023621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a49cf-964a-4d23-a993-0eacdf1e5e10 None admin] PATH_INFO: `/v3/services/image` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:26.024283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a49cf-964a-4d23-a993-0eacdf1e5e10 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:26.025749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a49cf-964a-4d23-a993-0eacdf1e5e10 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-4c7a49cf-964a-4d23-a993-0eacdf1e5e10 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:26.039026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:26.040098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:26.046257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 527/1056] 10.222.0.23 () {56 vars in 1105 bytes} [Tue Jul 12 13:15:25 2022] GET /identity/v3/services/image => generated 86 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:26.047069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685d37-df0a-4f5b-b73b-31723929928c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:26.153702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0685d37-df0a-4f5b-b73b-31723929928c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:26.188973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8290c96-534c-4b7a-87c2-0afb38d1b3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:26.189682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8290c96-534c-4b7a-87c2-0afb38d1b3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:26.189974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8290c96-534c-4b7a-87c2-0afb38d1b3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:26.190255 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8290c96-534c-4b7a-87c2-0afb38d1b3a4 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:26.190973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8290c96-534c-4b7a-87c2-0afb38d1b3a4 None admin] RBAC: Adding query filter params (name=image) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:26.191284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8290c96-534c-4b7a-87c2-0afb38d1b3a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:26.192663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8290c96-534c-4b7a-87c2-0afb38d1b3a4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:26.197831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 530/1057] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:26 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:26.203894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a49cf-964a-4d23-a993-0eacdf1e5e10 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:26.313772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a49cf-964a-4d23-a993-0eacdf1e5e10 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:26.348261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b22729-b855-4ebf-897d-6f8a2dea9457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:26.348929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b22729-b855-4ebf-897d-6f8a2dea9457 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:26.349221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b22729-b855-4ebf-897d-6f8a2dea9457 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:26.349532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b22729-b855-4ebf-897d-6f8a2dea9457 None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:26.350251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b22729-b855-4ebf-897d-6f8a2dea9457 None admin] RBAC: Adding query filter params (type=image) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:26.350563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b22729-b855-4ebf-897d-6f8a2dea9457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:26.351930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b22729-b855-4ebf-897d-6f8a2dea9457 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:26.358225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 528/1058] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:26 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:26.363843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8290c96-534c-4b7a-87c2-0afb38d1b3a4 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:26.398176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:26.475201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8290c96-534c-4b7a-87c2-0afb38d1b3a4 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:26.509564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa211c02-b019-4521-a4e4-7dff7cd5c36f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:26.510229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa211c02-b019-4521-a4e4-7dff7cd5c36f None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:26.510520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa211c02-b019-4521-a4e4-7dff7cd5c36f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:26.510803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa211c02-b019-4521-a4e4-7dff7cd5c36f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:26.511552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa211c02-b019-4521-a4e4-7dff7cd5c36f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:26.512928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa211c02-b019-4521-a4e4-7dff7cd5c36f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:26.541403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 531/1059] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 12 13:15:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:27.090000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62d3fb-95ad-4e47-99e0-6a6b8cc7a38e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:27.090375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62d3fb-95ad-4e47-99e0-6a6b8cc7a38e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:27.090575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d62d3fb-95ad-4e47-99e0-6a6b8cc7a38e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:27.091971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 529/1060] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:27.097891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a5697-3b5a-4b5f-bcdb-f79c8a2c72f6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:27.098188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a5697-3b5a-4b5f-bcdb-f79c8a2c72f6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:27.098470 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8a5697-3b5a-4b5f-bcdb-f79c8a2c72f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:27.115580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fe8a5697-3b5a-4b5f-bcdb-f79c8a2c72f6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:27.116776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fe8a5697-3b5a-4b5f-bcdb-f79c8a2c72f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:27.190377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 532/1061] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:27.212313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62d3fb-95ad-4e47-99e0-6a6b8cc7a38e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:27.253153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:27.329034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d62d3fb-95ad-4e47-99e0-6a6b8cc7a38e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:27.363953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3930388-1730-49f4-83ad-a8c4e9c74e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:27.364710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3930388-1730-49f4-83ad-a8c4e9c74e84 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:27.364999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3930388-1730-49f4-83ad-a8c4e9c74e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:27.365284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3930388-1730-49f4-83ad-a8c4e9c74e84 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:27.376485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3930388-1730-49f4-83ad-a8c4e9c74e84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-c3930388-1730-49f4-83ad-a8c4e9c74e84 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:27.384771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:27.385718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:27.392317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 530/1062] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 12 13:15:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:27.394542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a5697-3b5a-4b5f-bcdb-f79c8a2c72f6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:27.407372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:27.502440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8a5697-3b5a-4b5f-bcdb-f79c8a2c72f6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:27.538256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4a25-0147-4762-b5b3-eeaebe81452b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:27.538937 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4a25-0147-4762-b5b3-eeaebe81452b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:27.539226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4a25-0147-4762-b5b3-eeaebe81452b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:27.539508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f4a25-0147-4762-b5b3-eeaebe81452b None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:27.540262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f4a25-0147-4762-b5b3-eeaebe81452b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:27.540575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f4a25-0147-4762-b5b3-eeaebe81452b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:27.542077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f4a25-0147-4762-b5b3-eeaebe81452b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:27.555123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 533/1063] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 12 13:15:27 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:27.556718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3930388-1730-49f4-83ad-a8c4e9c74e84 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:27.651670 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:27.662346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3930388-1730-49f4-83ad-a8c4e9c74e84 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:27.696890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590fe14-d603-426e-989f-3a0538c5da4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:27.697712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590fe14-d603-426e-989f-3a0538c5da4d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:27.697995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590fe14-d603-426e-989f-3a0538c5da4d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:27.698265 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f590fe14-d603-426e-989f-3a0538c5da4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:27.708361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590fe14-d603-426e-989f-3a0538c5da4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:27.709830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f590fe14-d603-426e-989f-3a0538c5da4d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-f590fe14-d603-426e-989f-3a0538c5da4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:27.727245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:27.728190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:27.734360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:27.734360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:27.734360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:27.734360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:27.734360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 531/1064] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:27.736069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4a25-0147-4762-b5b3-eeaebe81452b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:27.806254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:27.837028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f4a25-0147-4762-b5b3-eeaebe81452b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:27.869878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70616d1c-1509-4034-919a-addd59a1d055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:27.870542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70616d1c-1509-4034-919a-addd59a1d055 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:27.870827 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70616d1c-1509-4034-919a-addd59a1d055 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:27.871105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70616d1c-1509-4034-919a-addd59a1d055 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:27.871831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70616d1c-1509-4034-919a-addd59a1d055 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:27.872141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70616d1c-1509-4034-919a-addd59a1d055 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:27.874170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70616d1c-1509-4034-919a-addd59a1d055 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:27.889417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 534/1065] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:27.892052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590fe14-d603-426e-989f-3a0538c5da4d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:27.955980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4d7eb-60b0-4e08-bf2b-1b3860b7dbba None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:27.956193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4d7eb-60b0-4e08-bf2b-1b3860b7dbba None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:27.956481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4d7eb-60b0-4e08-bf2b-1b3860b7dbba None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:27.957873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 535/1066] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:27.962744 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b986f-c9e1-4730-a43c-108809e4032e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:27.963032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b986f-c9e1-4730-a43c-108809e4032e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:27.963312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b986f-c9e1-4730-a43c-108809e4032e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:27.980187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c47b986f-c9e1-4730-a43c-108809e4032e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:27.981360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c47b986f-c9e1-4730-a43c-108809e4032e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:28.009912 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f590fe14-d603-426e-989f-3a0538c5da4d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.047630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea360c-83c4-4919-bd2a-1e6be6a1ed7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.048425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea360c-83c4-4919-bd2a-1e6be6a1ed7b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.048716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea360c-83c4-4919-bd2a-1e6be6a1ed7b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.048979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ea360c-83c4-4919-bd2a-1e6be6a1ed7b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.054196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 536/1067] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:28.056841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea360c-83c4-4919-bd2a-1e6be6a1ed7b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:28.058309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ea360c-83c4-4919-bd2a-1e6be6a1ed7b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-34ea360c-83c4-4919-bd2a-1e6be6a1ed7b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:15:28.073076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:28.074215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:28.080543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 532/1068] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 12 13:15:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:28.081129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b986f-c9e1-4730-a43c-108809e4032e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.083255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea360c-83c4-4919-bd2a-1e6be6a1ed7b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.087278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:28.143693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:28.184423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b986f-c9e1-4730-a43c-108809e4032e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.195665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ea360c-83c4-4919-bd2a-1e6be6a1ed7b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.221237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b750e-83d3-40b3-aca3-07857714cdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.222090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b750e-83d3-40b3-aca3-07857714cdef None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.222372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b750e-83d3-40b3-aca3-07857714cdef None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.222650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5b750e-83d3-40b3-aca3-07857714cdef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.234224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f2ce4-849d-4a9e-97f4-b4b2bdcc17dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.234697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5b750e-83d3-40b3-aca3-07857714cdef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:28.234889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f2ce4-849d-4a9e-97f4-b4b2bdcc17dc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.235123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f2ce4-849d-4a9e-97f4-b4b2bdcc17dc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.235398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4f2ce4-849d-4a9e-97f4-b4b2bdcc17dc None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.236244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f2ce4-849d-4a9e-97f4-b4b2bdcc17dc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:28.236555 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f2ce4-849d-4a9e-97f4-b4b2bdcc17dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:28.238022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4f2ce4-849d-4a9e-97f4-b4b2bdcc17dc None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-8a5b750e-83d3-40b3-aca3-07857714cdef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:28.242389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:28.243315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:28.244299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 537/1069] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:28.250267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b750e-83d3-40b3-aca3-07857714cdef None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.267200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 533/1070] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:28.268248 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f2ce4-849d-4a9e-97f4-b4b2bdcc17dc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.330490 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:28.368036 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5b750e-83d3-40b3-aca3-07857714cdef None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.369667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4f2ce4-849d-4a9e-97f4-b4b2bdcc17dc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.393282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:28.407137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535007a-1390-4a1c-a83d-d007dd2326f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.407935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535007a-1390-4a1c-a83d-d007dd2326f0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.408214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535007a-1390-4a1c-a83d-d007dd2326f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.408485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535007a-1390-4a1c-a83d-d007dd2326f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.409703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe11c0f-a7a1-48ae-94b4-96ebc9bce8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.411394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe11c0f-a7a1-48ae-94b4-96ebc9bce8c2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.411830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe11c0f-a7a1-48ae-94b4-96ebc9bce8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.412256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe11c0f-a7a1-48ae-94b4-96ebc9bce8c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.413220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe11c0f-a7a1-48ae-94b4-96ebc9bce8c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:28.413699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe11c0f-a7a1-48ae-94b4-96ebc9bce8c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:28.415721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe11c0f-a7a1-48ae-94b4-96ebc9bce8c2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:28.418675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535007a-1390-4a1c-a83d-d007dd2326f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:28.420097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535007a-1390-4a1c-a83d-d007dd2326f0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:28.426822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 538/1071] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:28.432493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe11c0f-a7a1-48ae-94b4-96ebc9bce8c2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-1535007a-1390-4a1c-a83d-d007dd2326f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:28.439729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:28.440792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:28.441895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:28.441895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:28.441895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:28.441895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:28.441895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 534/1072] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:28.446696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535007a-1390-4a1c-a83d-d007dd2326f0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.550167 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe11c0f-a7a1-48ae-94b4-96ebc9bce8c2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.552226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535007a-1390-4a1c-a83d-d007dd2326f0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.567304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:28.585755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4353940-101a-470e-aae0-d575da9739d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.586718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4353940-101a-470e-aae0-d575da9739d5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.587100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4353940-101a-470e-aae0-d575da9739d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.587370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4353940-101a-470e-aae0-d575da9739d5 None admin] PATH_INFO: `/v3/users/81829245596a48f4ad5a323ffe74478f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.588530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4353940-101a-470e-aae0-d575da9739d5 None admin] RBAC: Authorizing `identity:get_user(user_id=81829245596a48f4ad5a323ffe74478f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:28.590000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4353940-101a-470e-aae0-d575da9739d5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:28.591184 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23784fb6-26d0-4cb9-9622-8df83cdd29ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.591814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 539/1073] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/users/81829245596a48f4ad5a323ffe74478f => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:28.597235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23784fb6-26d0-4cb9-9622-8df83cdd29ec None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.597561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23784fb6-26d0-4cb9-9622-8df83cdd29ec None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.597772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4353940-101a-470e-aae0-d575da9739d5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.597870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23784fb6-26d0-4cb9-9622-8df83cdd29ec None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.598579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23784fb6-26d0-4cb9-9622-8df83cdd29ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:28.598889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23784fb6-26d0-4cb9-9622-8df83cdd29ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:28.600257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23784fb6-26d0-4cb9-9622-8df83cdd29ec None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:28.619104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 535/1074] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:28.619104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d12af-7f2c-4323-99b1-440fcea38f62 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.619104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d12af-7f2c-4323-99b1-440fcea38f62 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.619104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814d12af-7f2c-4323-99b1-440fcea38f62 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.619104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 536/1075] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:28.620180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d12af-7f2c-4323-99b1-440fcea38f62 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.658062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:28.707032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4353940-101a-470e-aae0-d575da9739d5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.720723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814d12af-7f2c-4323-99b1-440fcea38f62 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.740472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb3ceee-5add-4437-a01d-9043bc43863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.741331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3ceee-5add-4437-a01d-9043bc43863f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.741648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3ceee-5add-4437-a01d-9043bc43863f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.741934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb3ceee-5add-4437-a01d-9043bc43863f None admin] PATH_INFO: `/v3/projects/0743d4c307104b9eac562dd17591266f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.743139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3ceee-5add-4437-a01d-9043bc43863f None admin] RBAC: Authorizing `identity:get_project(project_id=0743d4c307104b9eac562dd17591266f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:28.744565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb3ceee-5add-4437-a01d-9043bc43863f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:28.746289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 540/1076] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/projects/0743d4c307104b9eac562dd17591266f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:28.747499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b690b1-a9b9-4896-937a-a84dd0564571 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.747790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b690b1-a9b9-4896-937a-a84dd0564571 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.748071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b690b1-a9b9-4896-937a-a84dd0564571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.755570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eb003-c597-4920-a785-d18e775aefff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.756365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eb003-c597-4920-a785-d18e775aefff None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.756651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eb003-c597-4920-a785-d18e775aefff None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.756927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5eb003-c597-4920-a785-d18e775aefff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.763688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5eb003-c597-4920-a785-d18e775aefff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:28.765748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5eb003-c597-4920-a785-d18e775aefff None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:28.766132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-41b690b1-a9b9-4896-937a-a84dd0564571 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:28.767320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-41b690b1-a9b9-4896-937a-a84dd0564571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3d5eb003-c597-4920-a785-d18e775aefff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:15:28.780613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:28.781622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:28.782514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:15:28.782514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:15:28.782514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:15:28.782514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:15:28.782514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:15:28.782514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:15:28.782514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:28.782514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:28.782514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 537/1077] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:28.782994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eb003-c597-4920-a785-d18e775aefff None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.816350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:28.840763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 541/1078] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:28.841818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b690b1-a9b9-4896-937a-a84dd0564571 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.885390 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5eb003-c597-4920-a785-d18e775aefff None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:28.893125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:28.920332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6c8dc-e173-4fd0-92ad-667058708f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:28.921236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6c8dc-e173-4fd0-92ad-667058708f99 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:28.921557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6c8dc-e173-4fd0-92ad-667058708f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:28.921835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c6c8dc-e173-4fd0-92ad-667058708f99 None admin] PATH_INFO: `/v3/projects/0743d4c307104b9eac562dd17591266f/users/81829245596a48f4ad5a323ffe74478f/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:28.923820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6c8dc-e173-4fd0-92ad-667058708f99 None admin] RBAC: Authorizing `identity:create_grant(project_id=0743d4c307104b9eac562dd17591266f, user_id=81829245596a48f4ad5a323ffe74478f, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:28.925260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c6c8dc-e173-4fd0-92ad-667058708f99 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:28.939237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 538/1079] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:15:28 2022] PUT /identity/v3/projects/0743d4c307104b9eac562dd17591266f/users/81829245596a48f4ad5a323ffe74478f/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:15:28.940025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6c8dc-e173-4fd0-92ad-667058708f99 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:28.957811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b690b1-a9b9-4896-937a-a84dd0564571 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:29.000056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b127c8-a05b-4e6b-92c4-415fff1866c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:29.001025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b127c8-a05b-4e6b-92c4-415fff1866c6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:29.001310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b127c8-a05b-4e6b-92c4-415fff1866c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:29.001610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b127c8-a05b-4e6b-92c4-415fff1866c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:29.002421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b127c8-a05b-4e6b-92c4-415fff1866c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:29.002735 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b127c8-a05b-4e6b-92c4-415fff1866c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:29.004212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b127c8-a05b-4e6b-92c4-415fff1866c6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:29.018311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 542/1080] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:29.024517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b127c8-a05b-4e6b-92c4-415fff1866c6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:29.058826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:29.068251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c6c8dc-e173-4fd0-92ad-667058708f99 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:29.109345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f2263-ac79-4178-afc4-579f1a235175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:29.110462 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f2263-ac79-4178-afc4-579f1a235175 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:29.110462 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f2263-ac79-4178-afc4-579f1a235175 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:29.110653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f2263-ac79-4178-afc4-579f1a235175 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:29.148002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b127c8-a05b-4e6b-92c4-415fff1866c6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:29.159766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:29.186463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416903e-3893-4e24-9522-b2da184976f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:29.187613 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416903e-3893-4e24-9522-b2da184976f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:29.187894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416903e-3893-4e24-9522-b2da184976f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:29.188171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a416903e-3893-4e24-9522-b2da184976f0 None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/1cc0050f14664630a36814ca670f124f/roles/4c2749d472714b3c85b29f3732da518e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:29.189912 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f2263-ac79-4178-afc4-579f1a235175 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:29.190518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416903e-3893-4e24-9522-b2da184976f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=1cc0050f14664630a36814ca670f124f, role_id=4c2749d472714b3c85b29f3732da518e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:29.191911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f2263-ac79-4178-afc4-579f1a235175 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:29.192022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a416903e-3893-4e24-9522-b2da184976f0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:29.217968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 543/1081] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:15:29 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/1cc0050f14664630a36814ca670f124f/roles/4c2749d472714b3c85b29f3732da518e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:15:29.295861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:29.305837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 539/1082] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:29.313111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416903e-3893-4e24-9522-b2da184976f0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:29.423565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a416903e-3893-4e24-9522-b2da184976f0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:29.458124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58d6fe6-f010-4e04-9329-2db8d31bc7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:29.458921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d6fe6-f010-4e04-9329-2db8d31bc7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:29.459205 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d6fe6-f010-4e04-9329-2db8d31bc7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:29.459480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58d6fe6-f010-4e04-9329-2db8d31bc7f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:29.470140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d6fe6-f010-4e04-9329-2db8d31bc7f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:29.471589 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58d6fe6-f010-4e04-9329-2db8d31bc7f4 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-d58d6fe6-f010-4e04-9329-2db8d31bc7f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:29.488508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:29.489767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:29.490670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:29.490670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:29.490670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:29.490670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:29.491199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 544/1083] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:29.495931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f2263-ac79-4178-afc4-579f1a235175 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:29.598590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f2263-ac79-4178-afc4-579f1a235175 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:29.632498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fed504-33a0-4622-a160-fe800446bec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:29.633178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fed504-33a0-4622-a160-fe800446bec2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:29.633472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fed504-33a0-4622-a160-fe800446bec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:29.633750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fed504-33a0-4622-a160-fe800446bec2 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:29.634475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fed504-33a0-4622-a160-fe800446bec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:29.634783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fed504-33a0-4622-a160-fe800446bec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:29.636136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fed504-33a0-4622-a160-fe800446bec2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:29.648188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 540/1084] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:30.981306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cfd65-941f-45f8-a56b-a210c9182475 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:30.981880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cfd65-941f-45f8-a56b-a210c9182475 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:30.981990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2cfd65-941f-45f8-a56b-a210c9182475 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:30.983291 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 545/1085] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:30.989068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474a677-a9ca-43fd-b106-39467062a465 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:30.989528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474a677-a9ca-43fd-b106-39467062a465 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:30.989667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474a677-a9ca-43fd-b106-39467062a465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.006216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8474a677-a9ca-43fd-b106-39467062a465 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:31.007406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8474a677-a9ca-43fd-b106-39467062a465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:31.055521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:31.074260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 541/1086] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:31.097648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cfd65-941f-45f8-a56b-a210c9182475 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:31.122372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:31.199624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2cfd65-941f-45f8-a56b-a210c9182475 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:31.232071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b412e7-bab3-4aa3-9887-09498c3402ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:31.233479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b412e7-bab3-4aa3-9887-09498c3402ae None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:31.233765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b412e7-bab3-4aa3-9887-09498c3402ae None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:31.234030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b412e7-bab3-4aa3-9887-09498c3402ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.244507 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b412e7-bab3-4aa3-9887-09498c3402ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-00b412e7-bab3-4aa3-9887-09498c3402ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:31.252869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:31.253883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:31.254734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 546/1087] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:15:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:31.260103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474a677-a9ca-43fd-b106-39467062a465 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:31.343971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14c03d-8e4d-42fc-b7fc-2adb47453fc6 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:31.344252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14c03d-8e4d-42fc-b7fc-2adb47453fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:31.344531 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee14c03d-8e4d-42fc-b7fc-2adb47453fc6 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.345934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 547/1088] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:15:31.350762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10121b8-ee96-48c5-803f-3956dfe8309b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:31.351042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10121b8-ee96-48c5-803f-3956dfe8309b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:31.351312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10121b8-ee96-48c5-803f-3956dfe8309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.357256 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474a677-a9ca-43fd-b106-39467062a465 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:31.367742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a10121b8-ee96-48c5-803f-3956dfe8309b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:31.368926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a10121b8-ee96-48c5-803f-3956dfe8309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:31.395713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0992e-f6bf-4631-a141-e4b526f20eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:31.396381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0992e-f6bf-4631-a141-e4b526f20eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:31.396664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0992e-f6bf-4631-a141-e4b526f20eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:31.396941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a0992e-f6bf-4631-a141-e4b526f20eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.397723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0992e-f6bf-4631-a141-e4b526f20eb0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:31.398029 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0992e-f6bf-4631-a141-e4b526f20eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:31.399419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a0992e-f6bf-4631-a141-e4b526f20eb0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:31.407451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 542/1089] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:15:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:31.413004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0992e-f6bf-4631-a141-e4b526f20eb0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:31.438062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 548/1090] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:15:31.461875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10121b8-ee96-48c5-803f-3956dfe8309b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:31.467993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:31.476922 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:31.528866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a0992e-f6bf-4631-a141-e4b526f20eb0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:31.564187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10121b8-ee96-48c5-803f-3956dfe8309b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:31.565024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bec2b8-4cc9-4aea-bd99-7f3bcc89f56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:31.565945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bec2b8-4cc9-4aea-bd99-7f3bcc89f56d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:31.566238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bec2b8-4cc9-4aea-bd99-7f3bcc89f56d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:31.566520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bec2b8-4cc9-4aea-bd99-7f3bcc89f56d None admin] PATH_INFO: `/v3/users/81829245596a48f4ad5a323ffe74478f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.567733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bec2b8-4cc9-4aea-bd99-7f3bcc89f56d None admin] RBAC: Authorizing `identity:get_user(user_id=81829245596a48f4ad5a323ffe74478f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:31.569170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bec2b8-4cc9-4aea-bd99-7f3bcc89f56d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:31.570840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 543/1091] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:31 2022] GET /identity/v3/users/81829245596a48f4ad5a323ffe74478f => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:31.576815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bec2b8-4cc9-4aea-bd99-7f3bcc89f56d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:31.598854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca87e05-03c1-40f6-b41e-510673ba41ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:31.599660 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca87e05-03c1-40f6-b41e-510673ba41ef None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:31.599947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca87e05-03c1-40f6-b41e-510673ba41ef None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:31.600223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca87e05-03c1-40f6-b41e-510673ba41ef None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.611607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca87e05-03c1-40f6-b41e-510673ba41ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6ca87e05-03c1-40f6-b41e-510673ba41ef None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:31.619854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:31.620934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:31.621863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 549/1092] 10.222.0.23 () {56 vars in 1120 bytes} [Tue Jul 12 13:15:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:31.627281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca87e05-03c1-40f6-b41e-510673ba41ef None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:31.676931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bec2b8-4cc9-4aea-bd99-7f3bcc89f56d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:31.691090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:31.695921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:31.711455 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b43f0-587b-464e-ac57-e57e2e833af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:31.712098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b43f0-587b-464e-ac57-e57e2e833af3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:31.712380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b43f0-587b-464e-ac57-e57e2e833af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:31.712655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b43f0-587b-464e-ac57-e57e2e833af3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.726756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca87e05-03c1-40f6-b41e-510673ba41ef None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:31.767751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1e939-3420-4c0f-9f5e-776eaa0f62d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:31.768421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1e939-3420-4c0f-9f5e-776eaa0f62d5 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:31.768708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1e939-3420-4c0f-9f5e-776eaa0f62d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:31.768988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b1e939-3420-4c0f-9f5e-776eaa0f62d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.769778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1e939-3420-4c0f-9f5e-776eaa0f62d5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:31.770090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1e939-3420-4c0f-9f5e-776eaa0f62d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:31.771462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b1e939-3420-4c0f-9f5e-776eaa0f62d5 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:31.779457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 550/1093] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 12 13:15:31 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:31.783613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b43f0-587b-464e-ac57-e57e2e833af3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:31.785136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b43f0-587b-464e-ac57-e57e2e833af3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:31.785544 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1e939-3420-4c0f-9f5e-776eaa0f62d5 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:31.881693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 544/1094] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:31.883203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b1e939-3420-4c0f-9f5e-776eaa0f62d5 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:31.887431 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b43f0-587b-464e-ac57-e57e2e833af3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:31.901882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:31.917618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b1315-b603-48d5-bd90-b10141a89da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:31.918414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b1315-b603-48d5-bd90-b10141a89da7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:31.918689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b1315-b603-48d5-bd90-b10141a89da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:31.918962 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b1315-b603-48d5-bd90-b10141a89da7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:31.920916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:31.931231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b1315-b603-48d5-bd90-b10141a89da7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:31.932753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25b1315-b603-48d5-bd90-b10141a89da7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c25b1315-b603-48d5-bd90-b10141a89da7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:31.949845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:31.950825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:31.952578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:31.952578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:31.952578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:31.952578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:31.952578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 551/1095] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:31.957316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b1315-b603-48d5-bd90-b10141a89da7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:31.987759 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b43f0-587b-464e-ac57-e57e2e833af3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:32.028558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc72d0-2e3a-4ebb-8000-ba9ed2d9a84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:32.030279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc72d0-2e3a-4ebb-8000-ba9ed2d9a84d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:32.030591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc72d0-2e3a-4ebb-8000-ba9ed2d9a84d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:32.030937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfc72d0-2e3a-4ebb-8000-ba9ed2d9a84d None admin] PATH_INFO: `/v3/projects/0743d4c307104b9eac562dd17591266f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:32.032208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc72d0-2e3a-4ebb-8000-ba9ed2d9a84d None admin] RBAC: Authorizing `identity:get_project(project_id=0743d4c307104b9eac562dd17591266f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:32.033738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cfc72d0-2e3a-4ebb-8000-ba9ed2d9a84d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:32.037540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 545/1096] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:15:31 2022] GET /identity/v3/projects/0743d4c307104b9eac562dd17591266f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:32.041449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc72d0-2e3a-4ebb-8000-ba9ed2d9a84d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:32.067961 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b1315-b603-48d5-bd90-b10141a89da7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:32.107293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad208-6810-4df1-941a-259067bd3a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:32.108009 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad208-6810-4df1-941a-259067bd3a84 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:32.108465 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad208-6810-4df1-941a-259067bd3a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:32.108752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bad208-6810-4df1-941a-259067bd3a84 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:32.109505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bad208-6810-4df1-941a-259067bd3a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:32.109818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bad208-6810-4df1-941a-259067bd3a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:32.111196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bad208-6810-4df1-941a-259067bd3a84 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:32.124725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 552/1097] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:32.130189 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad208-6810-4df1-941a-259067bd3a84 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:32.136823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:32.157204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfc72d0-2e3a-4ebb-8000-ba9ed2d9a84d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:32.182146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:32.193394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e28540-31f3-40be-bc3e-6eda4e695dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:32.194179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e28540-31f3-40be-bc3e-6eda4e695dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:32.194523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e28540-31f3-40be-bc3e-6eda4e695dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:32.194806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e28540-31f3-40be-bc3e-6eda4e695dc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:32.195639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e28540-31f3-40be-bc3e-6eda4e695dc3 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=0743d4c307104b9eac562dd17591266f, user.id=81829245596a48f4ad5a323ffe74478f) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:32.195945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e28540-31f3-40be-bc3e-6eda4e695dc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:32.197366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e28540-31f3-40be-bc3e-6eda4e695dc3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:32.203045 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 546/1098] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:32 2022] GET /identity/v3/role_assignments?user.id=81829245596a48f4ad5a323ffe74478f&scope.project.id=0743d4c307104b9eac562dd17591266f&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:32.240677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bad208-6810-4df1-941a-259067bd3a84 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:32.278949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddcb8e-2f00-4d90-9f75-04c846e1eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:32.280499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddcb8e-2f00-4d90-9f75-04c846e1eb06 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:32.280784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddcb8e-2f00-4d90-9f75-04c846e1eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:32.281126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ddcb8e-2f00-4d90-9f75-04c846e1eb06 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:32.289539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddcb8e-2f00-4d90-9f75-04c846e1eb06 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:32.291291 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ddcb8e-2f00-4d90-9f75-04c846e1eb06 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-40ddcb8e-2f00-4d90-9f75-04c846e1eb06 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:15:32.307942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:32.308956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:32.310433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 553/1099] 10.222.0.23 () {56 vars in 1093 bytes} [Tue Jul 12 13:15:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:32.316221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e28540-31f3-40be-bc3e-6eda4e695dc3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:32.422774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e28540-31f3-40be-bc3e-6eda4e695dc3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:32.456691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3bec0-1b9c-4569-9986-45e55c10f37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:32.457356 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3bec0-1b9c-4569-9986-45e55c10f37c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:32.457660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3bec0-1b9c-4569-9986-45e55c10f37c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:32.457935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c3bec0-1b9c-4569-9986-45e55c10f37c None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:32.458772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3bec0-1b9c-4569-9986-45e55c10f37c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:32.459080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3bec0-1b9c-4569-9986-45e55c10f37c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:32.460494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c3bec0-1b9c-4569-9986-45e55c10f37c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:32.477667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 547/1100] 10.222.0.23 () {56 vars in 1133 bytes} [Tue Jul 12 13:15:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:32.484138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddcb8e-2f00-4d90-9f75-04c846e1eb06 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:32.584951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ddcb8e-2f00-4d90-9f75-04c846e1eb06 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:32.592168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:32.618605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8c281-0057-4002-9a5e-c3b558f05ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:32.619252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8c281-0057-4002-9a5e-c3b558f05ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:32.619540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8c281-0057-4002-9a5e-c3b558f05ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:32.619812 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f8c281-0057-4002-9a5e-c3b558f05ad7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:32.684743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8c281-0057-4002-9a5e-c3b558f05ad7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:32.687052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f8c281-0057-4002-9a5e-c3b558f05ad7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:32.781850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 554/1101] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:32.787578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3bec0-1b9c-4569-9986-45e55c10f37c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:32.791308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:32.884943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c3bec0-1b9c-4569-9986-45e55c10f37c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:32.924039 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05f3ae-0dd9-4d81-84b4-9a77990a430b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:32.924835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f3ae-0dd9-4d81-84b4-9a77990a430b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:32.925122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f3ae-0dd9-4d81-84b4-9a77990a430b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:32.925419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05f3ae-0dd9-4d81-84b4-9a77990a430b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:32.935359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05f3ae-0dd9-4d81-84b4-9a77990a430b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:32.937702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05f3ae-0dd9-4d81-84b4-9a77990a430b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-0d05f3ae-0dd9-4d81-84b4-9a77990a430b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:15:32.955735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:32.956873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:32.958153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:15:32.958153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:15:32.958153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:15:32.958153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:32.958153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 548/1102] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:32.962584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8c281-0057-4002-9a5e-c3b558f05ad7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:32.971283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:33.061459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f8c281-0057-4002-9a5e-c3b558f05ad7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:33.095119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c2cc3-f9c5-4ec1-bdef-be0dbdb73fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:33.095783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c2cc3-f9c5-4ec1-bdef-be0dbdb73fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:33.096072 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c2cc3-f9c5-4ec1-bdef-be0dbdb73fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:33.096354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145c2cc3-f9c5-4ec1-bdef-be0dbdb73fa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:33.097083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c2cc3-f9c5-4ec1-bdef-be0dbdb73fa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:33.097418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c2cc3-f9c5-4ec1-bdef-be0dbdb73fa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:33.098795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145c2cc3-f9c5-4ec1-bdef-be0dbdb73fa1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:33.110204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 555/1103] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:15:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:33.115823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05f3ae-0dd9-4d81-84b4-9a77990a430b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:33.171755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:33.213761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05f3ae-0dd9-4d81-84b4-9a77990a430b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:33.250748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fabc74-0c5e-46d8-8539-621746344295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:33.251553 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fabc74-0c5e-46d8-8539-621746344295 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:33.251845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fabc74-0c5e-46d8-8539-621746344295 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:33.252128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fabc74-0c5e-46d8-8539-621746344295 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:33.258860 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fabc74-0c5e-46d8-8539-621746344295 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:33.260308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fabc74-0c5e-46d8-8539-621746344295 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-d0fabc74-0c5e-46d8-8539-621746344295 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:15:33.274504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:33.275397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:33.276325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:15:33.276325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:15:33.276325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:15:33.276325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:15:33.276325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:15:33.276325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:15:33.276325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:15:33.276325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:33.276325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 549/1104] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:15:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:15:33.282792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c2cc3-f9c5-4ec1-bdef-be0dbdb73fa1 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:33.350668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:33.386369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145c2cc3-f9c5-4ec1-bdef-be0dbdb73fa1 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:33.420271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1193b1cf-0cef-4c38-a371-4abf6658ef98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:33.421044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193b1cf-0cef-4c38-a371-4abf6658ef98 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:33.421334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193b1cf-0cef-4c38-a371-4abf6658ef98 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:33.421638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193b1cf-0cef-4c38-a371-4abf6658ef98 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:33.422476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193b1cf-0cef-4c38-a371-4abf6658ef98 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:33.422741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193b1cf-0cef-4c38-a371-4abf6658ef98 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:33.424168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1193b1cf-0cef-4c38-a371-4abf6658ef98 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:33.435887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 556/1105] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:15:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:33.442112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fabc74-0c5e-46d8-8539-621746344295 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:33.541359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fabc74-0c5e-46d8-8539-621746344295 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:33.556586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:33.574316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c92154-7e13-4847-8beb-8450a919df10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:33.574963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c92154-7e13-4847-8beb-8450a919df10 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:33.575242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c92154-7e13-4847-8beb-8450a919df10 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:33.575517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c92154-7e13-4847-8beb-8450a919df10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:33.576853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c92154-7e13-4847-8beb-8450a919df10 None admin] RBAC: Adding query filter params (role.id=4c2749d472714b3c85b29f3732da518e, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=1cc0050f14664630a36814ca670f124f) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:33.577170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c92154-7e13-4847-8beb-8450a919df10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:33.578614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c92154-7e13-4847-8beb-8450a919df10 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:33.584205 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 550/1106] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:33 2022] GET /identity/v3/role_assignments?user.id=1cc0050f14664630a36814ca670f124f&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=4c2749d472714b3c85b29f3732da518e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:34.306733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c1dc8-5d09-419f-baf7-a0178e7c404b None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:34.307241 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c1dc8-5d09-419f-baf7-a0178e7c404b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:34.307363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c1dc8-5d09-419f-baf7-a0178e7c404b None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:34.308659 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 557/1107] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:34.315004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a14eb-f9b6-4395-8d12-323a25c5a9e1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:34.315445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a14eb-f9b6-4395-8d12-323a25c5a9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:34.315576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a14eb-f9b6-4395-8d12-323a25c5a9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:34.333605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3a4a14eb-f9b6-4395-8d12-323a25c5a9e1 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:34.334814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3a4a14eb-f9b6-4395-8d12-323a25c5a9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:34.404108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 551/1108] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:34.429986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c1dc8-5d09-419f-baf7-a0178e7c404b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:34.533607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c1dc8-5d09-419f-baf7-a0178e7c404b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:34.557765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:34.567463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e074e9-80c7-42a4-a967-6e3771d1dad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:34.568260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e074e9-80c7-42a4-a967-6e3771d1dad2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:34.568548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e074e9-80c7-42a4-a967-6e3771d1dad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:34.568828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e074e9-80c7-42a4-a967-6e3771d1dad2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:34.570036 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e074e9-80c7-42a4-a967-6e3771d1dad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:34.571479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e074e9-80c7-42a4-a967-6e3771d1dad2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:34.573092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 558/1109] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:34.579426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a14eb-f9b6-4395-8d12-323a25c5a9e1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:34.674605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a14eb-f9b6-4395-8d12-323a25c5a9e1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:34.684753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:34.708709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb339e8-f874-48da-85f1-72396bec8f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:34.709362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb339e8-f874-48da-85f1-72396bec8f24 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:34.709675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb339e8-f874-48da-85f1-72396bec8f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:34.709950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb339e8-f874-48da-85f1-72396bec8f24 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:34.710658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb339e8-f874-48da-85f1-72396bec8f24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:34.712073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb339e8-f874-48da-85f1-72396bec8f24 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:34.730996 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 552/1110] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:15:34 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:36.767749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a028c7-d52e-493a-8593-f9737c257f97 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:36.768296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a028c7-d52e-493a-8593-f9737c257f97 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:36.768296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a028c7-d52e-493a-8593-f9737c257f97 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:36.770216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 559/1111] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:36.775420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6fa9a-c274-4dd5-9ec8-c8b8c3bd1eac None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:36.775828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6fa9a-c274-4dd5-9ec8-c8b8c3bd1eac None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:36.775981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa6fa9a-c274-4dd5-9ec8-c8b8c3bd1eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:36.792934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3fa6fa9a-c274-4dd5-9ec8-c8b8c3bd1eac None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:36.794152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3fa6fa9a-c274-4dd5-9ec8-c8b8c3bd1eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:36.858689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 553/1112] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:36.881431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a028c7-d52e-493a-8593-f9737c257f97 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:36.980067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a028c7-d52e-493a-8593-f9737c257f97 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:37.012499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b23d9-0817-4fe7-b53c-cdd86f065244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:37.013344 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b23d9-0817-4fe7-b53c-cdd86f065244 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:37.013716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b23d9-0817-4fe7-b53c-cdd86f065244 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:37.014029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00b23d9-0817-4fe7-b53c-cdd86f065244 None admin] PATH_INFO: `/v3/roles/3b6f494c47a6407284d1d344940e34cd` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:37.015476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b23d9-0817-4fe7-b53c-cdd86f065244 None admin] RBAC: Authorizing `identity:get_role(role_id=3b6f494c47a6407284d1d344940e34cd)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:37.016831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00b23d9-0817-4fe7-b53c-cdd86f065244 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:37.018263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 560/1113] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:36 2022] GET /identity/v3/roles/3b6f494c47a6407284d1d344940e34cd => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:37.023877 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6fa9a-c274-4dd5-9ec8-c8b8c3bd1eac None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:37.114634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:37.120713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa6fa9a-c274-4dd5-9ec8-c8b8c3bd1eac None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:37.155157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147ab8f-9157-4480-b755-98e739d08803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:37.156078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147ab8f-9157-4480-b755-98e739d08803 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:37.156365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147ab8f-9157-4480-b755-98e739d08803 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:37.156642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147ab8f-9157-4480-b755-98e739d08803 None admin] PATH_INFO: `/v3/users/2711ad6b2b43403abc44b25e8e34ab43` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:37.169090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147ab8f-9157-4480-b755-98e739d08803 None admin] RBAC: Authorizing `identity:get_user(user_id=2711ad6b2b43403abc44b25e8e34ab43)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:37.170714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147ab8f-9157-4480-b755-98e739d08803 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:37.172294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 554/1114] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:37 2022] GET /identity/v3/users/2711ad6b2b43403abc44b25e8e34ab43 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:37.178080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b23d9-0817-4fe7-b53c-cdd86f065244 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:37.239070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:37.276693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00b23d9-0817-4fe7-b53c-cdd86f065244 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:37.309231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d827714-6dec-4a5c-968f-38ee7859d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:37.309897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d827714-6dec-4a5c-968f-38ee7859d4fa None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:37.310181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d827714-6dec-4a5c-968f-38ee7859d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:37.310453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d827714-6dec-4a5c-968f-38ee7859d4fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:37.374365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d827714-6dec-4a5c-968f-38ee7859d4fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:37.375830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d827714-6dec-4a5c-968f-38ee7859d4fa None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:37.431884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:37.470617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 561/1115] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:37.476391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147ab8f-9157-4480-b755-98e739d08803 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:37.579080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147ab8f-9157-4480-b755-98e739d08803 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:37.610635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b263c40a-eadb-44af-9993-a974d6583327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:37.611481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263c40a-eadb-44af-9993-a974d6583327 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:37.611761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263c40a-eadb-44af-9993-a974d6583327 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:37.612032 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b263c40a-eadb-44af-9993-a974d6583327 None admin] PATH_INFO: `/v3/projects/0743d4c307104b9eac562dd17591266f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:37.613189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263c40a-eadb-44af-9993-a974d6583327 None admin] RBAC: Authorizing `identity:get_project(project_id=0743d4c307104b9eac562dd17591266f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:37.614634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b263c40a-eadb-44af-9993-a974d6583327 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:37.616295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 555/1116] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:15:37 2022] GET /identity/v3/projects/0743d4c307104b9eac562dd17591266f => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:37.622364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d827714-6dec-4a5c-968f-38ee7859d4fa None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:37.718424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d827714-6dec-4a5c-968f-38ee7859d4fa None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:37.750883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dc9192-4d84-4b27-8fd9-8070b62f822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:37.751518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc9192-4d84-4b27-8fd9-8070b62f822f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:37.751800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc9192-4d84-4b27-8fd9-8070b62f822f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:37.752075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dc9192-4d84-4b27-8fd9-8070b62f822f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:37.752895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc9192-4d84-4b27-8fd9-8070b62f822f None admin] RBAC: Adding query filter params (role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=0743d4c307104b9eac562dd17591266f, user.id=2711ad6b2b43403abc44b25e8e34ab43) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:37.753204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc9192-4d84-4b27-8fd9-8070b62f822f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:37.754634 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dc9192-4d84-4b27-8fd9-8070b62f822f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:37.759894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 562/1117] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:37 2022] GET /identity/v3/role_assignments?user.id=2711ad6b2b43403abc44b25e8e34ab43&scope.project.id=0743d4c307104b9eac562dd17591266f&role.id=3b6f494c47a6407284d1d344940e34cd => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:39.769681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540bd2-7cfc-4b00-b18e-407cbbff42c2 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:39.770234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540bd2-7cfc-4b00-b18e-407cbbff42c2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:39.770234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540bd2-7cfc-4b00-b18e-407cbbff42c2 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:39.772242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 556/1118] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:39.778289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e42304-d3fb-42b6-b8ae-266043ca217b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:39.778743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e42304-d3fb-42b6-b8ae-266043ca217b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:39.778837 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e42304-d3fb-42b6-b8ae-266043ca217b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:39.796208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b2e42304-d3fb-42b6-b8ae-266043ca217b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:39.797441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b2e42304-d3fb-42b6-b8ae-266043ca217b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:39.816798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:39.861750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 563/1119] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:39.884114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540bd2-7cfc-4b00-b18e-407cbbff42c2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:39.908886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:39.983741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67540bd2-7cfc-4b00-b18e-407cbbff42c2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:40.015150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7a874-d955-42e6-9659-062533f99972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:40.015985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7a874-d955-42e6-9659-062533f99972 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:40.016264 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7a874-d955-42e6-9659-062533f99972 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:40.016539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7a874-d955-42e6-9659-062533f99972 None admin] PATH_INFO: `/v3/roles/3b6f494c47a6407284d1d344940e34cd` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:40.018009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7a874-d955-42e6-9659-062533f99972 None admin] RBAC: Authorizing `identity:get_role(role_id=3b6f494c47a6407284d1d344940e34cd)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:40.019358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7a874-d955-42e6-9659-062533f99972 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:40.020739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 557/1120] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:39 2022] GET /identity/v3/roles/3b6f494c47a6407284d1d344940e34cd => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:40.026495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e42304-d3fb-42b6-b8ae-266043ca217b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:40.125037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e42304-d3fb-42b6-b8ae-266043ca217b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:40.160297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e39c8-6aaa-419b-8d2a-dabe6430cb4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:40.161166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e39c8-6aaa-419b-8d2a-dabe6430cb4a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:40.161485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e39c8-6aaa-419b-8d2a-dabe6430cb4a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:40.161769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e39c8-6aaa-419b-8d2a-dabe6430cb4a None admin] PATH_INFO: `/v3/users/2711ad6b2b43403abc44b25e8e34ab43` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:40.162950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e39c8-6aaa-419b-8d2a-dabe6430cb4a None admin] RBAC: Authorizing `identity:get_user(user_id=2711ad6b2b43403abc44b25e8e34ab43)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:40.164384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9e39c8-6aaa-419b-8d2a-dabe6430cb4a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:40.166027 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 564/1121] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:40 2022] GET /identity/v3/users/2711ad6b2b43403abc44b25e8e34ab43 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:40.171892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7a874-d955-42e6-9659-062533f99972 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:40.258542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:40.268756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7a874-d955-42e6-9659-062533f99972 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:40.301761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3350c-d4cb-4764-b1ae-cd252367d7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:40.302563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3350c-d4cb-4764-b1ae-cd252367d7af None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:40.302827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3350c-d4cb-4764-b1ae-cd252367d7af None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:40.303100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3350c-d4cb-4764-b1ae-cd252367d7af None admin] PATH_INFO: `/v3/projects/0743d4c307104b9eac562dd17591266f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:40.304261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3350c-d4cb-4764-b1ae-cd252367d7af None admin] RBAC: Authorizing `identity:get_project(project_id=0743d4c307104b9eac562dd17591266f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:40.305709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3350c-d4cb-4764-b1ae-cd252367d7af None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:40.307369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 558/1122] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:15:40 2022] GET /identity/v3/projects/0743d4c307104b9eac562dd17591266f => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:40.313778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e39c8-6aaa-419b-8d2a-dabe6430cb4a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:40.338215 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:40.409097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9e39c8-6aaa-419b-8d2a-dabe6430cb4a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:40.442054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a96882-36f9-49da-80d0-1eda63105def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:40.442924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a96882-36f9-49da-80d0-1eda63105def None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:40.443202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a96882-36f9-49da-80d0-1eda63105def None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:40.443473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a96882-36f9-49da-80d0-1eda63105def None admin] PATH_INFO: `/v3/projects/0743d4c307104b9eac562dd17591266f/users/2711ad6b2b43403abc44b25e8e34ab43/roles/3b6f494c47a6407284d1d344940e34cd` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:40.445635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a96882-36f9-49da-80d0-1eda63105def None admin] RBAC: Authorizing `identity:create_grant(project_id=0743d4c307104b9eac562dd17591266f, user_id=2711ad6b2b43403abc44b25e8e34ab43, role_id=3b6f494c47a6407284d1d344940e34cd)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:40.447054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a96882-36f9-49da-80d0-1eda63105def None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:40.459740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 565/1123] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:15:40 2022] PUT /identity/v3/projects/0743d4c307104b9eac562dd17591266f/users/2711ad6b2b43403abc44b25e8e34ab43/roles/3b6f494c47a6407284d1d344940e34cd => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:15:42.451795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454e798-3c53-4563-a2f5-65dbe9272516 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:42.452340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454e798-3c53-4563-a2f5-65dbe9272516 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:42.452340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a454e798-3c53-4563-a2f5-65dbe9272516 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:42.453810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 559/1124] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:42.458803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7d50d-37be-407b-8ae8-fdde155742f3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:42.459229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7d50d-37be-407b-8ae8-fdde155742f3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:42.459358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7d50d-37be-407b-8ae8-fdde155742f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:42.476152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f2a7d50d-37be-407b-8ae8-fdde155742f3 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:42.477328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f2a7d50d-37be-407b-8ae8-fdde155742f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:42.541506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 566/1125] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:42.564824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454e798-3c53-4563-a2f5-65dbe9272516 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:42.663436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a454e798-3c53-4563-a2f5-65dbe9272516 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:42.698001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a72af3-f51d-4482-8ce2-8b9bf4d4bc77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:42.699428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a72af3-f51d-4482-8ce2-8b9bf4d4bc77 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:42.699880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a72af3-f51d-4482-8ce2-8b9bf4d4bc77 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:42.700320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a72af3-f51d-4482-8ce2-8b9bf4d4bc77 None admin] PATH_INFO: `/v3/roles/3b6f494c47a6407284d1d344940e34cd` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:42.703211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a72af3-f51d-4482-8ce2-8b9bf4d4bc77 None admin] RBAC: Authorizing `identity:get_role(role_id=3b6f494c47a6407284d1d344940e34cd)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:42.705289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a72af3-f51d-4482-8ce2-8b9bf4d4bc77 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:42.706685 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 560/1126] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:42 2022] GET /identity/v3/roles/3b6f494c47a6407284d1d344940e34cd => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:42.712457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7d50d-37be-407b-8ae8-fdde155742f3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:42.727565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:42.808513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7d50d-37be-407b-8ae8-fdde155742f3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:42.842033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bbd55-8dec-4ed1-9358-9faa5bfe3151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:42.843053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bbd55-8dec-4ed1-9358-9faa5bfe3151 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:42.843338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bbd55-8dec-4ed1-9358-9faa5bfe3151 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:42.843621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619bbd55-8dec-4ed1-9358-9faa5bfe3151 None admin] PATH_INFO: `/v3/users/2711ad6b2b43403abc44b25e8e34ab43` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:42.844798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bbd55-8dec-4ed1-9358-9faa5bfe3151 None admin] RBAC: Authorizing `identity:get_user(user_id=2711ad6b2b43403abc44b25e8e34ab43)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:42.846261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619bbd55-8dec-4ed1-9358-9faa5bfe3151 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:42.847861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 567/1127] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:42 2022] GET /identity/v3/users/2711ad6b2b43403abc44b25e8e34ab43 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:42.853660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a72af3-f51d-4482-8ce2-8b9bf4d4bc77 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:42.880308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:42.952634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a72af3-f51d-4482-8ce2-8b9bf4d4bc77 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:42.984800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6756d23-7df9-4122-8d1e-0f10afc189d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:42.985459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6756d23-7df9-4122-8d1e-0f10afc189d3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:42.985741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6756d23-7df9-4122-8d1e-0f10afc189d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:42.986015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6756d23-7df9-4122-8d1e-0f10afc189d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:43.050666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6756d23-7df9-4122-8d1e-0f10afc189d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:43.052107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6756d23-7df9-4122-8d1e-0f10afc189d3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:43.078245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:43.147601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 561/1128] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:43.153507 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bbd55-8dec-4ed1-9358-9faa5bfe3151 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:43.231892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:43.251560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619bbd55-8dec-4ed1-9358-9faa5bfe3151 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:43.283920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00695923-dc8b-45a9-b3b2-e43418a3e26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:43.284771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00695923-dc8b-45a9-b3b2-e43418a3e26b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:43.285051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00695923-dc8b-45a9-b3b2-e43418a3e26b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:43.285321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00695923-dc8b-45a9-b3b2-e43418a3e26b None admin] PATH_INFO: `/v3/projects/0743d4c307104b9eac562dd17591266f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:43.286503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00695923-dc8b-45a9-b3b2-e43418a3e26b None admin] RBAC: Authorizing `identity:get_project(project_id=0743d4c307104b9eac562dd17591266f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:43.287930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00695923-dc8b-45a9-b3b2-e43418a3e26b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:43.289642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 568/1129] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:15:43 2022] GET /identity/v3/projects/0743d4c307104b9eac562dd17591266f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:43.296303 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6756d23-7df9-4122-8d1e-0f10afc189d3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:43.393737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6756d23-7df9-4122-8d1e-0f10afc189d3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:43.417925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:43.427496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce84e4-0843-4d48-9ba0-3d97c24e2272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:43.428138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce84e4-0843-4d48-9ba0-3d97c24e2272 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:43.428421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce84e4-0843-4d48-9ba0-3d97c24e2272 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:43.428695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce84e4-0843-4d48-9ba0-3d97c24e2272 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:43.429553 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce84e4-0843-4d48-9ba0-3d97c24e2272 None admin] RBAC: Adding query filter params (role.id=3b6f494c47a6407284d1d344940e34cd, scope.project.id=0743d4c307104b9eac562dd17591266f, user.id=2711ad6b2b43403abc44b25e8e34ab43) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:43.429863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce84e4-0843-4d48-9ba0-3d97c24e2272 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:43.431283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce84e4-0843-4d48-9ba0-3d97c24e2272 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:43.437939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 562/1130] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:43 2022] GET /identity/v3/role_assignments?user.id=2711ad6b2b43403abc44b25e8e34ab43&scope.project.id=0743d4c307104b9eac562dd17591266f&role.id=3b6f494c47a6407284d1d344940e34cd => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:45.452096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f3b3c-6684-49ca-9fa5-baa1832bfd66 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:45.452708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f3b3c-6684-49ca-9fa5-baa1832bfd66 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:45.452823 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f3b3c-6684-49ca-9fa5-baa1832bfd66 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:45.454223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 569/1131] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:45.459312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0024ef-76c7-4669-83c6-a7eb29b79e9b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:45.459757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0024ef-76c7-4669-83c6-a7eb29b79e9b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:45.459887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0024ef-76c7-4669-83c6-a7eb29b79e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:45.476779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-db0024ef-76c7-4669-83c6-a7eb29b79e9b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:45.478009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-db0024ef-76c7-4669-83c6-a7eb29b79e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:45.542717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 563/1132] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:45.565878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f3b3c-6684-49ca-9fa5-baa1832bfd66 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:45.665784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f3b3c-6684-49ca-9fa5-baa1832bfd66 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:45.698286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b09c97-3a20-41ea-8306-ac8a0cb582ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:45.699229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b09c97-3a20-41ea-8306-ac8a0cb582ca None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:45.699513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b09c97-3a20-41ea-8306-ac8a0cb582ca None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:45.699788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b09c97-3a20-41ea-8306-ac8a0cb582ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:45.700943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b09c97-3a20-41ea-8306-ac8a0cb582ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:45.703063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b09c97-3a20-41ea-8306-ac8a0cb582ca None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:45.704665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 570/1133] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:45.710471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0024ef-76c7-4669-83c6-a7eb29b79e9b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:45.814280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0024ef-76c7-4669-83c6-a7eb29b79e9b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:45.834025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:45.847847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772fbc77-9f39-4a93-839b-8b627ab65939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:45.848501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fbc77-9f39-4a93-839b-8b627ab65939 None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:45.848787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fbc77-9f39-4a93-839b-8b627ab65939 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:45.849063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772fbc77-9f39-4a93-839b-8b627ab65939 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:45.849817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fbc77-9f39-4a93-839b-8b627ab65939 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:45.851246 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772fbc77-9f39-4a93-839b-8b627ab65939 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:45.875277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 564/1134] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:15:45 2022] POST /identity/v3/projects => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:47.932394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b90cf-4917-4133-ad40-56fbb19aaee5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:47.932934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b90cf-4917-4133-ad40-56fbb19aaee5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:47.932934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562b90cf-4917-4133-ad40-56fbb19aaee5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:47.934334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 571/1135] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:47.940373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6c300-2e86-44d4-b29b-b9173976a9fb None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:47.940796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6c300-2e86-44d4-b29b-b9173976a9fb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:47.940934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6c300-2e86-44d4-b29b-b9173976a9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:47.957427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f1b6c300-2e86-44d4-b29b-b9173976a9fb None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:47.958618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f1b6c300-2e86-44d4-b29b-b9173976a9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:48.022407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 565/1136] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:48.047619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b90cf-4917-4133-ad40-56fbb19aaee5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:48.066473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:48.146196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562b90cf-4917-4133-ad40-56fbb19aaee5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:48.179677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729c105-f7c6-43e9-b894-66ded2ca5fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:48.180452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729c105-f7c6-43e9-b894-66ded2ca5fdb None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:48.180734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729c105-f7c6-43e9-b894-66ded2ca5fdb None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:48.181008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729c105-f7c6-43e9-b894-66ded2ca5fdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:48.182176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729c105-f7c6-43e9-b894-66ded2ca5fdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:48.183644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729c105-f7c6-43e9-b894-66ded2ca5fdb None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:48.185180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 572/1137] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:15:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:48.191170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6c300-2e86-44d4-b29b-b9173976a9fb None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:48.286988 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6c300-2e86-44d4-b29b-b9173976a9fb None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:48.288449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:48.322570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f095f9-f8d6-4c10-b36c-d099443b763c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:48.323229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f095f9-f8d6-4c10-b36c-d099443b763c None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:48.323513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f095f9-f8d6-4c10-b36c-d099443b763c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:48.323789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f095f9-f8d6-4c10-b36c-d099443b763c None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:48.324503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f095f9-f8d6-4c10-b36c-d099443b763c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:48.325969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f095f9-f8d6-4c10-b36c-d099443b763c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:48.345441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 566/1138] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:15:48 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:15:50.390668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cc7fd-5b61-4ac8-b715-29b85fd0c69a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:50.391193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cc7fd-5b61-4ac8-b715-29b85fd0c69a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:50.391193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5cc7fd-5b61-4ac8-b715-29b85fd0c69a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:50.392561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 573/1139] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:50.397847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6f659-c88f-48c0-a453-4f6b25e46861 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:50.398266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6f659-c88f-48c0-a453-4f6b25e46861 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:50.398394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f6f659-c88f-48c0-a453-4f6b25e46861 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:50.414714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-88f6f659-c88f-48c0-a453-4f6b25e46861 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:50.415890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-88f6f659-c88f-48c0-a453-4f6b25e46861 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:50.481361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 567/1140] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:50.506066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cc7fd-5b61-4ac8-b715-29b85fd0c69a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:50.594737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:50.611899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5cc7fd-5b61-4ac8-b715-29b85fd0c69a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:50.646225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6de79c-a2b2-4606-bc51-ad6aa36de52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:50.647087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6de79c-a2b2-4606-bc51-ad6aa36de52c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:50.647420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6de79c-a2b2-4606-bc51-ad6aa36de52c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:50.647746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6de79c-a2b2-4606-bc51-ad6aa36de52c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:50.658290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6de79c-a2b2-4606-bc51-ad6aa36de52c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-0a6de79c-a2b2-4606-bc51-ad6aa36de52c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:50.666069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:50.667192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:50.668279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 574/1141] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:50.674209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6f659-c88f-48c0-a453-4f6b25e46861 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:50.761302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:50.773929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f6f659-c88f-48c0-a453-4f6b25e46861 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:50.808282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6687aed-ace0-4a12-bd24-cf285f76929c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:50.808944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6687aed-ace0-4a12-bd24-cf285f76929c None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:50.809230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6687aed-ace0-4a12-bd24-cf285f76929c None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:50.809543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6687aed-ace0-4a12-bd24-cf285f76929c None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:50.810302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6687aed-ace0-4a12-bd24-cf285f76929c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:50.811163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6687aed-ace0-4a12-bd24-cf285f76929c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:50.812790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6687aed-ace0-4a12-bd24-cf285f76929c None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:50.823696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 568/1142] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:15:50.829617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6de79c-a2b2-4606-bc51-ad6aa36de52c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:50.932295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6de79c-a2b2-4606-bc51-ad6aa36de52c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:50.966367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de4f79-2115-4466-a98a-210de0937f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:50.967349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de4f79-2115-4466-a98a-210de0937f76 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:50.967664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de4f79-2115-4466-a98a-210de0937f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:50.967939 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27de4f79-2115-4466-a98a-210de0937f76 None admin] PATH_INFO: `/v3/users/d336af6a0fdc4340853a6c9b2080c61e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:50.981340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de4f79-2115-4466-a98a-210de0937f76 None admin] RBAC: Authorizing `identity:get_user(user_id=d336af6a0fdc4340853a6c9b2080c61e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:50.983452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27de4f79-2115-4466-a98a-210de0937f76 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:50.985474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 575/1143] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:50 2022] GET /identity/v3/users/d336af6a0fdc4340853a6c9b2080c61e => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:50.991689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6687aed-ace0-4a12-bd24-cf285f76929c None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:51.095649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6687aed-ace0-4a12-bd24-cf285f76929c None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:51.128885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a43bd-4b9a-411f-867b-79b3c4448aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:51.129561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a43bd-4b9a-411f-867b-79b3c4448aae None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:51.129846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a43bd-4b9a-411f-867b-79b3c4448aae None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:51.130122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a43bd-4b9a-411f-867b-79b3c4448aae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:51.149360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:51.202621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a43bd-4b9a-411f-867b-79b3c4448aae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:51.204073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a43bd-4b9a-411f-867b-79b3c4448aae None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:51.301485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 569/1144] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:51.307244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de4f79-2115-4466-a98a-210de0937f76 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:51.311051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:51.404060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27de4f79-2115-4466-a98a-210de0937f76 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:51.438647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c613ca65-1aac-4d2f-8bf9-b808efd6a1ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:51.439507 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613ca65-1aac-4d2f-8bf9-b808efd6a1ca None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:51.439791 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613ca65-1aac-4d2f-8bf9-b808efd6a1ca None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:51.440065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c613ca65-1aac-4d2f-8bf9-b808efd6a1ca None admin] PATH_INFO: `/v3/projects/12803248ee1a41b7ab6928c2b6e20b07` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:51.441244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613ca65-1aac-4d2f-8bf9-b808efd6a1ca None admin] RBAC: Authorizing `identity:get_project(project_id=12803248ee1a41b7ab6928c2b6e20b07)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:51.442709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c613ca65-1aac-4d2f-8bf9-b808efd6a1ca None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:51.444417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 576/1145] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:15:51 2022] GET /identity/v3/projects/12803248ee1a41b7ab6928c2b6e20b07 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:51.450564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a43bd-4b9a-411f-867b-79b3c4448aae None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:51.501159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:51.549881 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a43bd-4b9a-411f-867b-79b3c4448aae None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:51.583118 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e161658f-8456-49a4-a341-a75b4c3da321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:51.583767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161658f-8456-49a4-a341-a75b4c3da321 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:51.584054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161658f-8456-49a4-a341-a75b4c3da321 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:51.584333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e161658f-8456-49a4-a341-a75b4c3da321 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:51.585163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161658f-8456-49a4-a341-a75b4c3da321 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=12803248ee1a41b7ab6928c2b6e20b07, user.id=d336af6a0fdc4340853a6c9b2080c61e) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:51.585505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161658f-8456-49a4-a341-a75b4c3da321 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:51.586928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e161658f-8456-49a4-a341-a75b4c3da321 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:51.592996 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 570/1146] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:51 2022] GET /identity/v3/role_assignments?user.id=d336af6a0fdc4340853a6c9b2080c61e&scope.project.id=12803248ee1a41b7ab6928c2b6e20b07&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:53.589768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817ab8b-39d5-45d1-a499-3c2d78fb4513 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:53.590283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817ab8b-39d5-45d1-a499-3c2d78fb4513 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:53.590283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817ab8b-39d5-45d1-a499-3c2d78fb4513 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:53.591595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 577/1147] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:53.597326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46501a78-65d8-468a-9d64-779d3474e9f4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:53.597791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46501a78-65d8-468a-9d64-779d3474e9f4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:53.597911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46501a78-65d8-468a-9d64-779d3474e9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:53.615492 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-46501a78-65d8-468a-9d64-779d3474e9f4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:53.616666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-46501a78-65d8-468a-9d64-779d3474e9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:53.681342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 571/1148] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:53.704485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817ab8b-39d5-45d1-a499-3c2d78fb4513 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:53.775790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:53.806545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817ab8b-39d5-45d1-a499-3c2d78fb4513 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:53.840553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24476829-e9c7-46a8-abc1-7cb3b70874f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:53.841284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24476829-e9c7-46a8-abc1-7cb3b70874f9 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:53.841599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24476829-e9c7-46a8-abc1-7cb3b70874f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:53.843046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24476829-e9c7-46a8-abc1-7cb3b70874f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:53.852913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24476829-e9c7-46a8-abc1-7cb3b70874f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-24476829-e9c7-46a8-abc1-7cb3b70874f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:53.861040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:53.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:15:53.863201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 578/1149] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:15:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:53.868249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46501a78-65d8-468a-9d64-779d3474e9f4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:53.912528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:53.963716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46501a78-65d8-468a-9d64-779d3474e9f4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:53.995860 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350cf89-ec4c-42b8-b6b5-e16d576666fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:53.996512 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350cf89-ec4c-42b8-b6b5-e16d576666fb None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:53.996794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350cf89-ec4c-42b8-b6b5-e16d576666fb None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:53.997066 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350cf89-ec4c-42b8-b6b5-e16d576666fb None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:53.997836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350cf89-ec4c-42b8-b6b5-e16d576666fb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:53.998687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350cf89-ec4c-42b8-b6b5-e16d576666fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:54.000043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e350cf89-ec4c-42b8-b6b5-e16d576666fb None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:54.007735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 572/1150] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:15:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:54.013586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24476829-e9c7-46a8-abc1-7cb3b70874f9 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:54.110904 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24476829-e9c7-46a8-abc1-7cb3b70874f9 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:54.137638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:54.143217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cbf94-67ff-4dce-bf4f-8dad675ada2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:54.144082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cbf94-67ff-4dce-bf4f-8dad675ada2d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:54.144369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cbf94-67ff-4dce-bf4f-8dad675ada2d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:54.144645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217cbf94-67ff-4dce-bf4f-8dad675ada2d None admin] PATH_INFO: `/v3/users/d336af6a0fdc4340853a6c9b2080c61e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:54.145852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cbf94-67ff-4dce-bf4f-8dad675ada2d None admin] RBAC: Authorizing `identity:get_user(user_id=d336af6a0fdc4340853a6c9b2080c61e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:54.147281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217cbf94-67ff-4dce-bf4f-8dad675ada2d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:54.148893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 579/1151] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:54 2022] GET /identity/v3/users/d336af6a0fdc4340853a6c9b2080c61e => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:54.155364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350cf89-ec4c-42b8-b6b5-e16d576666fb None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:54.253811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350cf89-ec4c-42b8-b6b5-e16d576666fb None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:54.260439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:54.286825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256b3185-8fdf-4181-8c27-6528755ec8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:54.287679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b3185-8fdf-4181-8c27-6528755ec8ee None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:54.287961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b3185-8fdf-4181-8c27-6528755ec8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:54.288234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256b3185-8fdf-4181-8c27-6528755ec8ee None admin] PATH_INFO: `/v3/projects/12803248ee1a41b7ab6928c2b6e20b07` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:54.289420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b3185-8fdf-4181-8c27-6528755ec8ee None admin] RBAC: Authorizing `identity:get_project(project_id=12803248ee1a41b7ab6928c2b6e20b07)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:54.290849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256b3185-8fdf-4181-8c27-6528755ec8ee None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:54.292524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 573/1152] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:15:54 2022] GET /identity/v3/projects/12803248ee1a41b7ab6928c2b6e20b07 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:54.299162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cbf94-67ff-4dce-bf4f-8dad675ada2d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:54.396933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217cbf94-67ff-4dce-bf4f-8dad675ada2d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:54.429051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597d1a4-948c-44d0-a5ac-c453edb2b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:54.429955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597d1a4-948c-44d0-a5ac-c453edb2b587 None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:54.430240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597d1a4-948c-44d0-a5ac-c453edb2b587 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:54.430518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597d1a4-948c-44d0-a5ac-c453edb2b587 None admin] PATH_INFO: `/v3/projects/12803248ee1a41b7ab6928c2b6e20b07/users/d336af6a0fdc4340853a6c9b2080c61e/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:54.432453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597d1a4-948c-44d0-a5ac-c453edb2b587 None admin] RBAC: Authorizing `identity:create_grant(project_id=12803248ee1a41b7ab6928c2b6e20b07, user_id=d336af6a0fdc4340853a6c9b2080c61e, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:54.433914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597d1a4-948c-44d0-a5ac-c453edb2b587 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:54.446730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 580/1153] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:15:54 2022] PUT /identity/v3/projects/12803248ee1a41b7ab6928c2b6e20b07/users/d336af6a0fdc4340853a6c9b2080c61e/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:15:56.452518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5b1eb-dc17-4a92-9614-ea62053406ad None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:56.453039 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5b1eb-dc17-4a92-9614-ea62053406ad None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:56.453151 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf5b1eb-dc17-4a92-9614-ea62053406ad None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:56.454454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 574/1154] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:56.459554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a45d5-a58c-414e-9fd3-664f892b6673 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:56.459946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a45d5-a58c-414e-9fd3-664f892b6673 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:56.460107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a45d5-a58c-414e-9fd3-664f892b6673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:56.477075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c87a45d5-a58c-414e-9fd3-664f892b6673 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:56.478365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c87a45d5-a58c-414e-9fd3-664f892b6673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:56.532067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:56.543906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 581/1155] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:56.567480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5b1eb-dc17-4a92-9614-ea62053406ad None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:56.666905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf5b1eb-dc17-4a92-9614-ea62053406ad None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:56.698971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4be3b-5ee2-464e-a915-a644f97db71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:56.699757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4be3b-5ee2-464e-a915-a644f97db71f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:56.700041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4be3b-5ee2-464e-a915-a644f97db71f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:56.700313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d4be3b-5ee2-464e-a915-a644f97db71f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:56.711751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d4be3b-5ee2-464e-a915-a644f97db71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-76d4be3b-5ee2-464e-a915-a644f97db71f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:15:56.720542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:15:56.721647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:15:56.722695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 575/1156] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:15:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:15:56.727613 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a45d5-a58c-414e-9fd3-664f892b6673 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:56.825401 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a45d5-a58c-414e-9fd3-664f892b6673 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:56.860171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191ec4b-2c64-4ef6-8abc-caa566b7c8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:56.860835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ec4b-2c64-4ef6-8abc-caa566b7c8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:56.861121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ec4b-2c64-4ef6-8abc-caa566b7c8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:56.861422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6191ec4b-2c64-4ef6-8abc-caa566b7c8b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:56.862172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191ec4b-2c64-4ef6-8abc-caa566b7c8b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:56.862481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191ec4b-2c64-4ef6-8abc-caa566b7c8b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:56.863855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6191ec4b-2c64-4ef6-8abc-caa566b7c8b7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:56.871668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 582/1157] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:15:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:56.877433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4be3b-5ee2-464e-a915-a644f97db71f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:56.911147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:56.975286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d4be3b-5ee2-464e-a915-a644f97db71f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:57.007857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915a5b3-0614-41d8-b8ef-4d27b3306f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:57.008710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915a5b3-0614-41d8-b8ef-4d27b3306f5a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:57.008998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915a5b3-0614-41d8-b8ef-4d27b3306f5a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:57.009283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915a5b3-0614-41d8-b8ef-4d27b3306f5a None admin] PATH_INFO: `/v3/users/d336af6a0fdc4340853a6c9b2080c61e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:57.010497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915a5b3-0614-41d8-b8ef-4d27b3306f5a None admin] RBAC: Authorizing `identity:get_user(user_id=d336af6a0fdc4340853a6c9b2080c61e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:57.011924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915a5b3-0614-41d8-b8ef-4d27b3306f5a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:57.013617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 576/1158] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:15:56 2022] GET /identity/v3/users/d336af6a0fdc4340853a6c9b2080c61e => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:57.019190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191ec4b-2c64-4ef6-8abc-caa566b7c8b7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:57.070325 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:57.117147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6191ec4b-2c64-4ef6-8abc-caa566b7c8b7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:57.148625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe64627-9178-4d89-a8c5-c3996bd17894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:57.149263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe64627-9178-4d89-a8c5-c3996bd17894 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:57.149578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe64627-9178-4d89-a8c5-c3996bd17894 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:57.149857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe64627-9178-4d89-a8c5-c3996bd17894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:57.215004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe64627-9178-4d89-a8c5-c3996bd17894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:57.216449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe64627-9178-4d89-a8c5-c3996bd17894 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:57.265620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:57.312388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 583/1159] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:15:57.318251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915a5b3-0614-41d8-b8ef-4d27b3306f5a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:57.413567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915a5b3-0614-41d8-b8ef-4d27b3306f5a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:57.414978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:57.448861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7463b3-a7d7-40ab-9661-eaa1c558eed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:57.449729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7463b3-a7d7-40ab-9661-eaa1c558eed8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:57.450010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7463b3-a7d7-40ab-9661-eaa1c558eed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:57.450284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7463b3-a7d7-40ab-9661-eaa1c558eed8 None admin] PATH_INFO: `/v3/projects/12803248ee1a41b7ab6928c2b6e20b07` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:57.451479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7463b3-a7d7-40ab-9661-eaa1c558eed8 None admin] RBAC: Authorizing `identity:get_project(project_id=12803248ee1a41b7ab6928c2b6e20b07)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:57.452896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7463b3-a7d7-40ab-9661-eaa1c558eed8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:57.454580 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 577/1160] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:15:57 2022] GET /identity/v3/projects/12803248ee1a41b7ab6928c2b6e20b07 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:57.460460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe64627-9178-4d89-a8c5-c3996bd17894 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:57.555955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe64627-9178-4d89-a8c5-c3996bd17894 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:57.589551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05892206-6d7e-44e0-8a50-04c733b6c0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:57.590191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05892206-6d7e-44e0-8a50-04c733b6c0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:57.590471 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05892206-6d7e-44e0-8a50-04c733b6c0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:57.590745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05892206-6d7e-44e0-8a50-04c733b6c0d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:57.591735 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05892206-6d7e-44e0-8a50-04c733b6c0d3 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=12803248ee1a41b7ab6928c2b6e20b07, user.id=d336af6a0fdc4340853a6c9b2080c61e) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:15:57.591970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05892206-6d7e-44e0-8a50-04c733b6c0d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:57.593534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05892206-6d7e-44e0-8a50-04c733b6c0d3 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:15:57.599176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 584/1161] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:15:57 2022] GET /identity/v3/role_assignments?user.id=d336af6a0fdc4340853a6c9b2080c61e&scope.project.id=12803248ee1a41b7ab6928c2b6e20b07&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:15:59.611910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c8e86-cf4a-439b-b45e-81ce9e8626af None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:59.612424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c8e86-cf4a-439b-b45e-81ce9e8626af None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:59.612424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48c8e86-cf4a-439b-b45e-81ce9e8626af None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:59.613774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 578/1162] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:15:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:15:59.618768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c67a4-c83a-4b8a-b59c-3470d444346c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:59.619182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c67a4-c83a-4b8a-b59c-3470d444346c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:59.619324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c67a4-c83a-4b8a-b59c-3470d444346c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:59.636426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5d4c67a4-c83a-4b8a-b59c-3470d444346c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:15:59.637674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5d4c67a4-c83a-4b8a-b59c-3470d444346c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:15:59.648200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:59.704722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 585/1163] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:15:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:15:59.727201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c8e86-cf4a-439b-b45e-81ce9e8626af None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:15:59.826550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48c8e86-cf4a-439b-b45e-81ce9e8626af None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:15:59.858991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29c07e-404c-4780-bdf7-d4796e6aad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:15:59.859627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29c07e-404c-4780-bdf7-d4796e6aad17 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:15:59.859911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29c07e-404c-4780-bdf7-d4796e6aad17 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:15:59.860187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29c07e-404c-4780-bdf7-d4796e6aad17 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:15:59.898015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:15:59.925925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29c07e-404c-4780-bdf7-d4796e6aad17 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:15:59.927368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29c07e-404c-4780-bdf7-d4796e6aad17 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:00.021147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 579/1164] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:15:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:16:00.027095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c67a4-c83a-4b8a-b59c-3470d444346c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:00.123424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c67a4-c83a-4b8a-b59c-3470d444346c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:00.156066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83788f6e-ddf5-4b5b-bbb9-eceaac5fba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:00.156796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83788f6e-ddf5-4b5b-bbb9-eceaac5fba2b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:00.157081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83788f6e-ddf5-4b5b-bbb9-eceaac5fba2b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:00.157360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83788f6e-ddf5-4b5b-bbb9-eceaac5fba2b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:00.163898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83788f6e-ddf5-4b5b-bbb9-eceaac5fba2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:00.165345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83788f6e-ddf5-4b5b-bbb9-eceaac5fba2b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-83788f6e-ddf5-4b5b-bbb9-eceaac5fba2b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:16:00.179581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:16:00.180788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:16:00.181871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 586/1165] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:16:00 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:16:00.187862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29c07e-404c-4780-bdf7-d4796e6aad17 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:00.255218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:00.285419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29c07e-404c-4780-bdf7-d4796e6aad17 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:00.318217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fa8a9d-a051-4085-b8d1-d6e648178857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:00.318869 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa8a9d-a051-4085-b8d1-d6e648178857 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:00.319152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa8a9d-a051-4085-b8d1-d6e648178857 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:00.319424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fa8a9d-a051-4085-b8d1-d6e648178857 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:00.320145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa8a9d-a051-4085-b8d1-d6e648178857 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:16:00.320449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa8a9d-a051-4085-b8d1-d6e648178857 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:00.321827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fa8a9d-a051-4085-b8d1-d6e648178857 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:00.329399 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 580/1166] 10.222.0.23 () {56 vars in 1115 bytes} [Tue Jul 12 13:16:00 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:02.347512 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f10186-1f9b-4654-a3ee-38554336afc7 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:02.348034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f10186-1f9b-4654-a3ee-38554336afc7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:02.348140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f10186-1f9b-4654-a3ee-38554336afc7 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:02.349463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 587/1167] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:16:02.354656 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1048c1-26c4-4137-bf86-97faa358e898 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:02.355052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1048c1-26c4-4137-bf86-97faa358e898 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:02.355222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1048c1-26c4-4137-bf86-97faa358e898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:02.372056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7b1048c1-26c4-4137-bf86-97faa358e898 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:16:02.373495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7b1048c1-26c4-4137-bf86-97faa358e898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:16:02.438857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 581/1168] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:16:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:16:02.462012 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f10186-1f9b-4654-a3ee-38554336afc7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:02.471734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:02.567484 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f10186-1f9b-4654-a3ee-38554336afc7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:02.602114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e2ac7-55fe-4ce3-b960-003260bf245e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:02.602775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e2ac7-55fe-4ce3-b960-003260bf245e None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:02.603063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e2ac7-55fe-4ce3-b960-003260bf245e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:02.603345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e2ac7-55fe-4ce3-b960-003260bf245e None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:02.604096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e2ac7-55fe-4ce3-b960-003260bf245e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:02.605506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e2ac7-55fe-4ce3-b960-003260bf245e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:02.620850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 588/1169] 10.222.0.23 () {60 vars in 1138 bytes} [Tue Jul 12 13:16:02 2022] POST /identity/v3/domains => generated 267 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:16:04.642281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d460f-3e93-432b-a99f-2b154538b557 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:04.642783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d460f-3e93-432b-a99f-2b154538b557 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:04.642783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d460f-3e93-432b-a99f-2b154538b557 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:04.644121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 582/1170] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:16:04.649302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cc5fd-ddb2-4649-9ba6-023da5300d91 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:04.649747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cc5fd-ddb2-4649-9ba6-023da5300d91 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:04.649898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99cc5fd-ddb2-4649-9ba6-023da5300d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:04.667086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c99cc5fd-ddb2-4649-9ba6-023da5300d91 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:16:04.668260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c99cc5fd-ddb2-4649-9ba6-023da5300d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:16:04.725156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:04.733542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 589/1171] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:16:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:16:04.756379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d460f-3e93-432b-a99f-2b154538b557 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:04.817643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:04.854564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d460f-3e93-432b-a99f-2b154538b557 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:04.887466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb074ce-42c0-4ab5-8ef9-dcc2d0f2f7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:04.888836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb074ce-42c0-4ab5-8ef9-dcc2d0f2f7fd None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:04.889121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb074ce-42c0-4ab5-8ef9-dcc2d0f2f7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:04.889421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb074ce-42c0-4ab5-8ef9-dcc2d0f2f7fd None admin] PATH_INFO: `/v3/domains/0ae964ce854a4cfc9e21b4f028490c6c` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:04.899536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb074ce-42c0-4ab5-8ef9-dcc2d0f2f7fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ae964ce854a4cfc9e21b4f028490c6c)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:04.900933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb074ce-42c0-4ab5-8ef9-dcc2d0f2f7fd None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:04.902572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 583/1172] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Jul 12 13:16:04 2022] GET /identity/v3/domains/0ae964ce854a4cfc9e21b4f028490c6c => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:04.908931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cc5fd-ddb2-4649-9ba6-023da5300d91 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:05.006890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99cc5fd-ddb2-4649-9ba6-023da5300d91 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:05.039492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86db5f-24f2-461b-a9dc-e0704e897bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:05.040142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86db5f-24f2-461b-a9dc-e0704e897bd6 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:05.040425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86db5f-24f2-461b-a9dc-e0704e897bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:05.040699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86db5f-24f2-461b-a9dc-e0704e897bd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:05.041416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86db5f-24f2-461b-a9dc-e0704e897bd6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:05.042838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86db5f-24f2-461b-a9dc-e0704e897bd6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:05.066775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 590/1173] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:16:04 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:16:07.107889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3653ff2-b5d3-41bf-a3de-d91d41147dab None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:07.108600 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3653ff2-b5d3-41bf-a3de-d91d41147dab None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:07.108841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3653ff2-b5d3-41bf-a3de-d91d41147dab None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:07.110673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 584/1174] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:16:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:16:07.115848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7225d78-ad5d-400d-871e-55bd88e30239 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:07.116265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7225d78-ad5d-400d-871e-55bd88e30239 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:07.116417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7225d78-ad5d-400d-871e-55bd88e30239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:07.133817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e7225d78-ad5d-400d-871e-55bd88e30239 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:16:07.135023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e7225d78-ad5d-400d-871e-55bd88e30239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:16:07.177489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:07.202380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 591/1175] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:16:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:16:07.226769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3653ff2-b5d3-41bf-a3de-d91d41147dab None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:07.327903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3653ff2-b5d3-41bf-a3de-d91d41147dab None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:07.349800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:07.360872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03c27c-c619-4626-afc9-54bc86c1c5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:07.361701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c27c-c619-4626-afc9-54bc86c1c5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:07.361988 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c27c-c619-4626-afc9-54bc86c1c5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:07.362266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03c27c-c619-4626-afc9-54bc86c1c5f0 None admin] PATH_INFO: `/v3/domains/0ae964ce854a4cfc9e21b4f028490c6c` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:07.363435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c27c-c619-4626-afc9-54bc86c1c5f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0ae964ce854a4cfc9e21b4f028490c6c)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:07.364864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03c27c-c619-4626-afc9-54bc86c1c5f0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:07.366476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 585/1176] 10.222.0.23 () {56 vars in 1183 bytes} [Tue Jul 12 13:16:07 2022] GET /identity/v3/domains/0ae964ce854a4cfc9e21b4f028490c6c => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:07.372097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7225d78-ad5d-400d-871e-55bd88e30239 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:07.467003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7225d78-ad5d-400d-871e-55bd88e30239 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:07.500593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a53af-5d34-449c-a443-c506b1487e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:07.501245 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a53af-5d34-449c-a443-c506b1487e64 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:07.501561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a53af-5d34-449c-a443-c506b1487e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:07.501839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a53af-5d34-449c-a443-c506b1487e64 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:07.502553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a53af-5d34-449c-a443-c506b1487e64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:07.503980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a53af-5d34-449c-a443-c506b1487e64 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:07.522753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 592/1177] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:16:07 2022] POST /identity/v3/users => generated 311 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:16:09.582456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618da39a-452a-43de-9cc7-61deeab46008 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:09.582937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618da39a-452a-43de-9cc7-61deeab46008 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:09.582937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618da39a-452a-43de-9cc7-61deeab46008 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:09.584296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 586/1178] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:16:09.589591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb391f0-5137-4bf5-9f01-a4a2e25909f7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:09.590002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb391f0-5137-4bf5-9f01-a4a2e25909f7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:09.590151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb391f0-5137-4bf5-9f01-a4a2e25909f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:09.609043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-eeb391f0-5137-4bf5-9f01-a4a2e25909f7 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:16:09.610477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-eeb391f0-5137-4bf5-9f01-a4a2e25909f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:16:09.636066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:09.676124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 593/1179] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:16:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:16:09.700160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618da39a-452a-43de-9cc7-61deeab46008 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:09.802402 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618da39a-452a-43de-9cc7-61deeab46008 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:09.836821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11ae7b-171a-4e3a-a8bd-bbdbd5e792cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:09.837615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11ae7b-171a-4e3a-a8bd-bbdbd5e792cc None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:09.837904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11ae7b-171a-4e3a-a8bd-bbdbd5e792cc None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:09.838186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11ae7b-171a-4e3a-a8bd-bbdbd5e792cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:09.848471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a11ae7b-171a-4e3a-a8bd-bbdbd5e792cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7a11ae7b-171a-4e3a-a8bd-bbdbd5e792cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:16:09.856079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:16:09.857277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:16:09.858791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 587/1180] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:16:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:16:09.863309 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb391f0-5137-4bf5-9f01-a4a2e25909f7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:09.962338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb391f0-5137-4bf5-9f01-a4a2e25909f7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:09.994679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50695d7-e14b-4375-8ef4-9ad9c470b690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:09.995338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50695d7-e14b-4375-8ef4-9ad9c470b690 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:09.995623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50695d7-e14b-4375-8ef4-9ad9c470b690 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:09.995897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50695d7-e14b-4375-8ef4-9ad9c470b690 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:09.996638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50695d7-e14b-4375-8ef4-9ad9c470b690 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:16:09.996944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50695d7-e14b-4375-8ef4-9ad9c470b690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:09.998945 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50695d7-e14b-4375-8ef4-9ad9c470b690 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:10.006708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 594/1181] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:16:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:10.012416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11ae7b-171a-4e3a-a8bd-bbdbd5e792cc None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:10.064149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:10.109824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11ae7b-171a-4e3a-a8bd-bbdbd5e792cc None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:10.142558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82aeb7-aded-40f8-b73a-8739befaf9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:10.143412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82aeb7-aded-40f8-b73a-8739befaf9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:10.143697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82aeb7-aded-40f8-b73a-8739befaf9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:10.143971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd82aeb7-aded-40f8-b73a-8739befaf9f2 None admin] PATH_INFO: `/v3/users/57bd4084386f4ba4ac8b8a7ccf9b62f0` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:10.156511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82aeb7-aded-40f8-b73a-8739befaf9f2 None admin] RBAC: Authorizing `identity:get_user(user_id=57bd4084386f4ba4ac8b8a7ccf9b62f0)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:10.157963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd82aeb7-aded-40f8-b73a-8739befaf9f2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:10.159564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 588/1182] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:16:10 2022] GET /identity/v3/users/57bd4084386f4ba4ac8b8a7ccf9b62f0 => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:10.165293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50695d7-e14b-4375-8ef4-9ad9c470b690 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:10.193229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:10.264244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50695d7-e14b-4375-8ef4-9ad9c470b690 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:10.298397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a2d35-1b5d-4519-8f18-47468a10a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:10.299030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a2d35-1b5d-4519-8f18-47468a10a89e None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:10.299313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a2d35-1b5d-4519-8f18-47468a10a89e None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:10.299587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649a2d35-1b5d-4519-8f18-47468a10a89e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:10.364604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a2d35-1b5d-4519-8f18-47468a10a89e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:10.366009 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649a2d35-1b5d-4519-8f18-47468a10a89e None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:10.390818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:10.461148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 595/1183] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:16:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:16:10.467226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82aeb7-aded-40f8-b73a-8739befaf9f2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:10.561751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd82aeb7-aded-40f8-b73a-8739befaf9f2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:10.572117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:10.598211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c4b92b-9dbb-4c6b-a104-1a5bfc7dce21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:10.599095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4b92b-9dbb-4c6b-a104-1a5bfc7dce21 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:10.599376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4b92b-9dbb-4c6b-a104-1a5bfc7dce21 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:10.599655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c4b92b-9dbb-4c6b-a104-1a5bfc7dce21 None admin] PATH_INFO: `/v3/projects/f9ba6e9f25c748a98521456b12023a7d` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:10.600821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4b92b-9dbb-4c6b-a104-1a5bfc7dce21 None admin] RBAC: Authorizing `identity:get_project(project_id=f9ba6e9f25c748a98521456b12023a7d)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:10.602273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c4b92b-9dbb-4c6b-a104-1a5bfc7dce21 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:10.603946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 589/1184] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:16:10 2022] GET /identity/v3/projects/f9ba6e9f25c748a98521456b12023a7d => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:10.611097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a2d35-1b5d-4519-8f18-47468a10a89e None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:10.707912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649a2d35-1b5d-4519-8f18-47468a10a89e None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:10.733226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:10.741890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5266e12-865a-462b-9674-af278667c3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:10.742533 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266e12-865a-462b-9674-af278667c3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:10.742813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266e12-865a-462b-9674-af278667c3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:10.743084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5266e12-865a-462b-9674-af278667c3d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:10.743897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266e12-865a-462b-9674-af278667c3d7 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=f9ba6e9f25c748a98521456b12023a7d, user.id=57bd4084386f4ba4ac8b8a7ccf9b62f0) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:16:10.744203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266e12-865a-462b-9674-af278667c3d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:10.745631 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5266e12-865a-462b-9674-af278667c3d7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:10.750870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 596/1185] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:16:10 2022] GET /identity/v3/role_assignments?user.id=57bd4084386f4ba4ac8b8a7ccf9b62f0&scope.project.id=f9ba6e9f25c748a98521456b12023a7d&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:12.744204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda0c3a-3f05-4469-b7c5-944256baa59c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:12.744737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda0c3a-3f05-4469-b7c5-944256baa59c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:12.744840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda0c3a-3f05-4469-b7c5-944256baa59c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:12.746190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 590/1186] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:16:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:16:12.751831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2186a87-cba1-4188-84c8-22cbdb3ebe5c None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:12.752217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2186a87-cba1-4188-84c8-22cbdb3ebe5c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:12.752405 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2186a87-cba1-4188-84c8-22cbdb3ebe5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:12.769346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c2186a87-cba1-4188-84c8-22cbdb3ebe5c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:16:12.770564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c2186a87-cba1-4188-84c8-22cbdb3ebe5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:16:12.835590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 597/1187] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:16:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:16:12.857717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda0c3a-3f05-4469-b7c5-944256baa59c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:12.954778 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda0c3a-3f05-4469-b7c5-944256baa59c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:12.986457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f3553-c573-400d-b3db-fa877565432e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:12.987235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f3553-c573-400d-b3db-fa877565432e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:12.987516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f3553-c573-400d-b3db-fa877565432e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:12.987790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2f3553-c573-400d-b3db-fa877565432e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:12.997479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2f3553-c573-400d-b3db-fa877565432e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-0d2f3553-c573-400d-b3db-fa877565432e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:16:13.005784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:16:13.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:16:13.008051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 591/1188] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:16:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:16:13.013052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2186a87-cba1-4188-84c8-22cbdb3ebe5c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:13.109726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2186a87-cba1-4188-84c8-22cbdb3ebe5c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:13.129860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:13.144021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340b64-a1a4-4cc8-b6a7-06060ee50700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:13.144685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340b64-a1a4-4cc8-b6a7-06060ee50700 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:13.144970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340b64-a1a4-4cc8-b6a7-06060ee50700 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:13.145246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84340b64-a1a4-4cc8-b6a7-06060ee50700 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:13.146026 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340b64-a1a4-4cc8-b6a7-06060ee50700 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:16:13.146343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340b64-a1a4-4cc8-b6a7-06060ee50700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:13.147703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84340b64-a1a4-4cc8-b6a7-06060ee50700 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:13.155627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 598/1189] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:16:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:13.161500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f3553-c573-400d-b3db-fa877565432e None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:13.201984 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:13.261295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2f3553-c573-400d-b3db-fa877565432e None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:13.293336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69995250-f0a9-4c86-8073-0fc0d71255bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:13.294233 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69995250-f0a9-4c86-8073-0fc0d71255bf None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:13.295032 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69995250-f0a9-4c86-8073-0fc0d71255bf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:13.295375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69995250-f0a9-4c86-8073-0fc0d71255bf None admin] PATH_INFO: `/v3/users/57bd4084386f4ba4ac8b8a7ccf9b62f0` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:13.296562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69995250-f0a9-4c86-8073-0fc0d71255bf None admin] RBAC: Authorizing `identity:get_user(user_id=57bd4084386f4ba4ac8b8a7ccf9b62f0)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:13.298042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69995250-f0a9-4c86-8073-0fc0d71255bf None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:13.299644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 592/1190] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:16:13 2022] GET /identity/v3/users/57bd4084386f4ba4ac8b8a7ccf9b62f0 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:13.305331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340b64-a1a4-4cc8-b6a7-06060ee50700 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:13.403078 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84340b64-a1a4-4cc8-b6a7-06060ee50700 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:13.436492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f7f472-bc1d-40f6-84d8-85b6fd420dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:13.437636 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7f472-bc1d-40f6-84d8-85b6fd420dd1 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:13.437924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7f472-bc1d-40f6-84d8-85b6fd420dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:13.438200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f7f472-bc1d-40f6-84d8-85b6fd420dd1 None admin] PATH_INFO: `/v3/projects/f9ba6e9f25c748a98521456b12023a7d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:13.439397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7f472-bc1d-40f6-84d8-85b6fd420dd1 None admin] RBAC: Authorizing `identity:get_project(project_id=f9ba6e9f25c748a98521456b12023a7d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:13.440824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f7f472-bc1d-40f6-84d8-85b6fd420dd1 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:13.442538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 599/1191] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:16:13 2022] GET /identity/v3/projects/f9ba6e9f25c748a98521456b12023a7d => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:13.448215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69995250-f0a9-4c86-8073-0fc0d71255bf None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:13.544027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69995250-f0a9-4c86-8073-0fc0d71255bf None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:13.551194 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:13.577188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fe67f-b86c-40ea-bfc7-20579370a3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:13.578106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe67f-b86c-40ea-bfc7-20579370a3f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:13.578394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe67f-b86c-40ea-bfc7-20579370a3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:13.578672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fe67f-b86c-40ea-bfc7-20579370a3f1 None admin] PATH_INFO: `/v3/projects/f9ba6e9f25c748a98521456b12023a7d/users/57bd4084386f4ba4ac8b8a7ccf9b62f0/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:13.580617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fe67f-b86c-40ea-bfc7-20579370a3f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9ba6e9f25c748a98521456b12023a7d, user_id=57bd4084386f4ba4ac8b8a7ccf9b62f0, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:13.582085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fe67f-b86c-40ea-bfc7-20579370a3f1 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:13.594475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 593/1192] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:16:13 2022] PUT /identity/v3/projects/f9ba6e9f25c748a98521456b12023a7d/users/57bd4084386f4ba4ac8b8a7ccf9b62f0/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:16:15.570715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3021-80e1-47db-8e8b-f920da077c86 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:15.571253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3021-80e1-47db-8e8b-f920da077c86 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:15.571253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c3021-80e1-47db-8e8b-f920da077c86 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:15.572553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 600/1193] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:16:15.578396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476daa7-f6cd-4d59-9f15-a7f0719ea04c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:15.578794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476daa7-f6cd-4d59-9f15-a7f0719ea04c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:15.578954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476daa7-f6cd-4d59-9f15-a7f0719ea04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:15.595723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4476daa7-f6cd-4d59-9f15-a7f0719ea04c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:16:15.596907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4476daa7-f6cd-4d59-9f15-a7f0719ea04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:16:15.660859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 594/1194] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:16:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:16:15.683984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c3021-80e1-47db-8e8b-f920da077c86 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:15.734395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:15.785150 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c3021-80e1-47db-8e8b-f920da077c86 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:15.817645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9b05b-bd14-4ca1-a27b-bb72a45aeb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:15.818428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9b05b-bd14-4ca1-a27b-bb72a45aeb68 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:15.818716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9b05b-bd14-4ca1-a27b-bb72a45aeb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:15.818997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9b05b-bd14-4ca1-a27b-bb72a45aeb68 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:15.830074 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9b05b-bd14-4ca1-a27b-bb72a45aeb68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-68a9b05b-bd14-4ca1-a27b-bb72a45aeb68 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:16:15.837746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:16:15.838807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:16:15.839766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 601/1195] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:16:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:16:15.845190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476daa7-f6cd-4d59-9f15-a7f0719ea04c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:15.934508 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:15.941543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476daa7-f6cd-4d59-9f15-a7f0719ea04c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:15.975156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebfae6-bd3e-4bb1-a95a-593eaf9286a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:15.975815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebfae6-bd3e-4bb1-a95a-593eaf9286a3 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:15.976099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebfae6-bd3e-4bb1-a95a-593eaf9286a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:15.976375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deebfae6-bd3e-4bb1-a95a-593eaf9286a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:15.977124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebfae6-bd3e-4bb1-a95a-593eaf9286a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:16:15.977458 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebfae6-bd3e-4bb1-a95a-593eaf9286a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:15.978824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deebfae6-bd3e-4bb1-a95a-593eaf9286a3 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:15.987337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 595/1196] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:16:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:15.992968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9b05b-bd14-4ca1-a27b-bb72a45aeb68 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:16.088058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9b05b-bd14-4ca1-a27b-bb72a45aeb68 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:16.099583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:16.122715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f61ae-a6ea-4ed1-8bdd-f598d89e389c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:16.123572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f61ae-a6ea-4ed1-8bdd-f598d89e389c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:16.123858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f61ae-a6ea-4ed1-8bdd-f598d89e389c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:16.124135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f61ae-a6ea-4ed1-8bdd-f598d89e389c None admin] PATH_INFO: `/v3/users/57bd4084386f4ba4ac8b8a7ccf9b62f0` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:16.125298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f61ae-a6ea-4ed1-8bdd-f598d89e389c None admin] RBAC: Authorizing `identity:get_user(user_id=57bd4084386f4ba4ac8b8a7ccf9b62f0)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:16.126745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f61ae-a6ea-4ed1-8bdd-f598d89e389c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:16.128348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 602/1197] 10.222.0.23 () {56 vars in 1177 bytes} [Tue Jul 12 13:16:15 2022] GET /identity/v3/users/57bd4084386f4ba4ac8b8a7ccf9b62f0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:16.133994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebfae6-bd3e-4bb1-a95a-593eaf9286a3 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:16.231171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deebfae6-bd3e-4bb1-a95a-593eaf9286a3 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:16.262739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40614d5-8189-4ff7-aa8e-97fbe6804975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:16.264014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40614d5-8189-4ff7-aa8e-97fbe6804975 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:16.264469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40614d5-8189-4ff7-aa8e-97fbe6804975 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:16.264751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40614d5-8189-4ff7-aa8e-97fbe6804975 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:16.290774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:16.329966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40614d5-8189-4ff7-aa8e-97fbe6804975 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:16.331399 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40614d5-8189-4ff7-aa8e-97fbe6804975 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:16.425803 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 596/1198] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:16:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:16:16.431653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f61ae-a6ea-4ed1-8bdd-f598d89e389c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:16.530819 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f61ae-a6ea-4ed1-8bdd-f598d89e389c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:16.564502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea97598-3af2-4a90-92ee-2f3a704c08b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:16.565355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97598-3af2-4a90-92ee-2f3a704c08b6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:16.565739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97598-3af2-4a90-92ee-2f3a704c08b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:16.566019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea97598-3af2-4a90-92ee-2f3a704c08b6 None admin] PATH_INFO: `/v3/projects/f9ba6e9f25c748a98521456b12023a7d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:16.567185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97598-3af2-4a90-92ee-2f3a704c08b6 None admin] RBAC: Authorizing `identity:get_project(project_id=f9ba6e9f25c748a98521456b12023a7d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:16.568619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea97598-3af2-4a90-92ee-2f3a704c08b6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:16.570331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 603/1199] 10.222.0.23 () {56 vars in 1186 bytes} [Tue Jul 12 13:16:16 2022] GET /identity/v3/projects/f9ba6e9f25c748a98521456b12023a7d => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:16.576373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40614d5-8189-4ff7-aa8e-97fbe6804975 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:16:16.632680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:16.673361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40614d5-8189-4ff7-aa8e-97fbe6804975 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:16:16.705630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b85eb2e-483b-4ffe-a996-38c1bca06a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:16:16.706214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85eb2e-483b-4ffe-a996-38c1bca06a2e None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:16:16.706499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85eb2e-483b-4ffe-a996-38c1bca06a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:16:16.706773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b85eb2e-483b-4ffe-a996-38c1bca06a2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:16:16.707603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85eb2e-483b-4ffe-a996-38c1bca06a2e None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=f9ba6e9f25c748a98521456b12023a7d, user.id=57bd4084386f4ba4ac8b8a7ccf9b62f0) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:16:16.707910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85eb2e-483b-4ffe-a996-38c1bca06a2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:16:16.709337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b85eb2e-483b-4ffe-a996-38c1bca06a2e None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:16:16.714956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 597/1200] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:16:16 2022] GET /identity/v3/role_assignments?user.id=57bd4084386f4ba4ac8b8a7ccf9b62f0&scope.project.id=f9ba6e9f25c748a98521456b12023a7d&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:16:26.559848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=77 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:16:26.712373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=39 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:11.814976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49f002-fe34-4891-ab84-fe18520c1ad1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:11.815560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49f002-fe34-4891-ab84-fe18520c1ad1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:11.815560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a49f002-fe34-4891-ab84-fe18520c1ad1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:11.817186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 604/1201] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:11.823488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975b92-77a2-4f13-a24c-c6b251a2fad9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:11.823488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975b92-77a2-4f13-a24c-c6b251a2fad9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:11.823488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2975b92-77a2-4f13-a24c-c6b251a2fad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:11.846708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f2975b92-77a2-4f13-a24c-c6b251a2fad9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:11.850277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f2975b92-77a2-4f13-a24c-c6b251a2fad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:11.935408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 598/1202] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 12 13:17:11.962903 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49f002-fe34-4891-ab84-fe18520c1ad1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:12.098201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a49f002-fe34-4891-ab84-fe18520c1ad1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:12.144387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7e69a-ea74-45ab-8ebc-bf88c94ac69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:12.145234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7e69a-ea74-45ab-8ebc-bf88c94ac69b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:12.145567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7e69a-ea74-45ab-8ebc-bf88c94ac69b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:12.145870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c7e69a-ea74-45ab-8ebc-bf88c94ac69b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:12.157354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7e69a-ea74-45ab-8ebc-bf88c94ac69b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:12.158967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c7e69a-ea74-45ab-8ebc-bf88c94ac69b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-c2c7e69a-ea74-45ab-8ebc-bf88c94ac69b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:12.187010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:12.188597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:12.189695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:12.189695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:12.189695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:12.189695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:12.194532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975b92-77a2-4f13-a24c-c6b251a2fad9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:12.196498 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 605/1203] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:12.328522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2975b92-77a2-4f13-a24c-c6b251a2fad9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:12.365563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:12.378715 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aadafe-0e84-4e80-83f1-2a290636a473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:12.379435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aadafe-0e84-4e80-83f1-2a290636a473 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:12.379751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aadafe-0e84-4e80-83f1-2a290636a473 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:12.380086 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aadafe-0e84-4e80-83f1-2a290636a473 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:12.380979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aadafe-0e84-4e80-83f1-2a290636a473 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:12.381319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aadafe-0e84-4e80-83f1-2a290636a473 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:12.382883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aadafe-0e84-4e80-83f1-2a290636a473 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:12.398678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 599/1204] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:12.404354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7e69a-ea74-45ab-8ebc-bf88c94ac69b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:12.471409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:12.536115 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c7e69a-ea74-45ab-8ebc-bf88c94ac69b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:12.581462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454f7b13-92c0-439f-80dd-a1936e3a1fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:12.582453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7b13-92c0-439f-80dd-a1936e3a1fbf None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:12.582720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7b13-92c0-439f-80dd-a1936e3a1fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:12.582979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454f7b13-92c0-439f-80dd-a1936e3a1fbf None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:12.583699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f7b13-92c0-439f-80dd-a1936e3a1fbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:12.585149 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454f7b13-92c0-439f-80dd-a1936e3a1fbf None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:12.608344 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 606/1205] 10.222.0.23 () {60 vars in 1132 bytes} [Tue Jul 12 13:17:12 2022] POST /identity/v3/users => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:17:14.781237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77b32a-0ff0-44f3-8b6a-0a09802c7b74 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:14.781838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77b32a-0ff0-44f3-8b6a-0a09802c7b74 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:14.781838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77b32a-0ff0-44f3-8b6a-0a09802c7b74 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:14.783182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 600/1206] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:14.788384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba2e3b-432a-426e-b37f-a2814404617f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:14.788809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba2e3b-432a-426e-b37f-a2814404617f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:14.788951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba2e3b-432a-426e-b37f-a2814404617f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:14.805983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4dba2e3b-432a-426e-b37f-a2814404617f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:14.807178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4dba2e3b-432a-426e-b37f-a2814404617f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:14.883033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 607/1207] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:14.907828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77b32a-0ff0-44f3-8b6a-0a09802c7b74 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:15.018408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77b32a-0ff0-44f3-8b6a-0a09802c7b74 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:15.053731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbe762-5551-451c-8769-2985070f63f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:15.054467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbe762-5551-451c-8769-2985070f63f0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:15.054758 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbe762-5551-451c-8769-2985070f63f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:15.055040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccbe762-5551-451c-8769-2985070f63f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:15.067349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccbe762-5551-451c-8769-2985070f63f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-bccbe762-5551-451c-8769-2985070f63f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:17:15.076165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:15.077465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:15.078510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 601/1208] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:17:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:17:15.083458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba2e3b-432a-426e-b37f-a2814404617f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:15.125441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:15.195225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba2e3b-432a-426e-b37f-a2814404617f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:15.234546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebbf3c-af9d-46bd-a327-e891b5017152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:15.235477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebbf3c-af9d-46bd-a327-e891b5017152 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:15.235770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebbf3c-af9d-46bd-a327-e891b5017152 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:15.236080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ebbf3c-af9d-46bd-a327-e891b5017152 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:15.236793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebbf3c-af9d-46bd-a327-e891b5017152 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:15.237103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebbf3c-af9d-46bd-a327-e891b5017152 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:15.238506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ebbf3c-af9d-46bd-a327-e891b5017152 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:15.251168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 608/1209] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:17:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:15.255511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbe762-5551-451c-8769-2985070f63f0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:15.313095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:15.369898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccbe762-5551-451c-8769-2985070f63f0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:15.406593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870baf5a-4613-4c65-8f78-cdd70e6a6ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:15.407522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870baf5a-4613-4c65-8f78-cdd70e6a6ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:15.407883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870baf5a-4613-4c65-8f78-cdd70e6a6ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:15.408233 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870baf5a-4613-4c65-8f78-cdd70e6a6ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:15.420145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870baf5a-4613-4c65-8f78-cdd70e6a6ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:15.421644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870baf5a-4613-4c65-8f78-cdd70e6a6ec4 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-870baf5a-4613-4c65-8f78-cdd70e6a6ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:15.444296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:15.445704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:15.446812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:15.446812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:15.446812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:15.446812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:15.446812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 602/1210] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:15.452654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebbf3c-af9d-46bd-a327-e891b5017152 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:15.568774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:15.572662 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ebbf3c-af9d-46bd-a327-e891b5017152 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:15.610878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8a439-eacc-4477-b55a-b0a0ae7244a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:15.611555 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8a439-eacc-4477-b55a-b0a0ae7244a2 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:15.611847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8a439-eacc-4477-b55a-b0a0ae7244a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:15.612129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8a439-eacc-4477-b55a-b0a0ae7244a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:15.612912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8a439-eacc-4477-b55a-b0a0ae7244a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:15.613227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8a439-eacc-4477-b55a-b0a0ae7244a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:15.614636 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8a439-eacc-4477-b55a-b0a0ae7244a2 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:15.632164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 609/1211] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:15.633918 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870baf5a-4613-4c65-8f78-cdd70e6a6ec4 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:15.748993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870baf5a-4613-4c65-8f78-cdd70e6a6ec4 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:15.750652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:15.788427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683364d3-5512-4715-ad01-28a2fe1bc10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:15.789582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683364d3-5512-4715-ad01-28a2fe1bc10d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:15.789869 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683364d3-5512-4715-ad01-28a2fe1bc10d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:15.790178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683364d3-5512-4715-ad01-28a2fe1bc10d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:15.799554 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683364d3-5512-4715-ad01-28a2fe1bc10d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:15.801001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683364d3-5512-4715-ad01-28a2fe1bc10d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-683364d3-5512-4715-ad01-28a2fe1bc10d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:17:15.820129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:15.821461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:15.822425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 603/1212] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:15.828445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8a439-eacc-4477-b55a-b0a0ae7244a2 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:15.971922 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8a439-eacc-4477-b55a-b0a0ae7244a2 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:16.024302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0558bd-3067-40b9-932c-96f49dc6b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:16.024987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0558bd-3067-40b9-932c-96f49dc6b5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:16.025278 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0558bd-3067-40b9-932c-96f49dc6b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:16.025586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0558bd-3067-40b9-932c-96f49dc6b5c6 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:16.026445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0558bd-3067-40b9-932c-96f49dc6b5c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:16.026758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0558bd-3067-40b9-932c-96f49dc6b5c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:16.029290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0558bd-3067-40b9-932c-96f49dc6b5c6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:16.051633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 610/1213] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 12 13:17:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:16.057334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683364d3-5512-4715-ad01-28a2fe1bc10d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:16.176782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683364d3-5512-4715-ad01-28a2fe1bc10d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:16.221538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93192fb-e8c1-4bb0-a19c-f92ad213ff82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:16.222363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93192fb-e8c1-4bb0-a19c-f92ad213ff82 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:16.222692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93192fb-e8c1-4bb0-a19c-f92ad213ff82 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:16.223105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93192fb-e8c1-4bb0-a19c-f92ad213ff82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:16.267758 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:16.316129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93192fb-e8c1-4bb0-a19c-f92ad213ff82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:16.317828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93192fb-e8c1-4bb0-a19c-f92ad213ff82 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:16.433487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 604/1214] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:17:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:17:16.445693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0558bd-3067-40b9-932c-96f49dc6b5c6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:16.470305 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:16.573366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0558bd-3067-40b9-932c-96f49dc6b5c6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:16.616730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ce2a4-6bdc-4068-a465-a50764e0bb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:16.617576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ce2a4-6bdc-4068-a465-a50764e0bb58 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:16.617867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ce2a4-6bdc-4068-a465-a50764e0bb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:16.618143 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ce2a4-6bdc-4068-a465-a50764e0bb58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:16.629185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ce2a4-6bdc-4068-a465-a50764e0bb58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:16.630867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ce2a4-6bdc-4068-a465-a50764e0bb58 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-588ce2a4-6bdc-4068-a465-a50764e0bb58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:16.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:16.652781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:16.659011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:16.659011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:16.659011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:16.659011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:16.659011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 611/1215] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:16.659487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93192fb-e8c1-4bb0-a19c-f92ad213ff82 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:16.739866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:16.791952 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93192fb-e8c1-4bb0-a19c-f92ad213ff82 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:16.835630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb219ab3-9c96-4a70-ac18-a4b9fbb8a64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:16.836400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb219ab3-9c96-4a70-ac18-a4b9fbb8a64d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:16.836755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb219ab3-9c96-4a70-ac18-a4b9fbb8a64d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:16.836992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb219ab3-9c96-4a70-ac18-a4b9fbb8a64d None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:16.837975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb219ab3-9c96-4a70-ac18-a4b9fbb8a64d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:16.838270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb219ab3-9c96-4a70-ac18-a4b9fbb8a64d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:16.839854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb219ab3-9c96-4a70-ac18-a4b9fbb8a64d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:16.852813 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 605/1216] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:17:16.859457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ce2a4-6bdc-4068-a465-a50764e0bb58 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:16.945431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:16.988656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ce2a4-6bdc-4068-a465-a50764e0bb58 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:17.037031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c27d1-f7a0-4457-b39b-db0ac59f1b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:17.037031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c27d1-f7a0-4457-b39b-db0ac59f1b53 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:17.037031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c27d1-f7a0-4457-b39b-db0ac59f1b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:17.037031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5c27d1-f7a0-4457-b39b-db0ac59f1b53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:17.045810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c27d1-f7a0-4457-b39b-db0ac59f1b53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:17.047371 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5c27d1-f7a0-4457-b39b-db0ac59f1b53 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-cd5c27d1-f7a0-4457-b39b-db0ac59f1b53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:17:17.065258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:17.066432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:17.067310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:17:17.067310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:17:17.067310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:17:17.067310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:17:17.067310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:17:17.067310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:17:17.067310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:17.067310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:17.071565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 612/1217] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:17:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:17.072823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb219ab3-9c96-4a70-ac18-a4b9fbb8a64d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:17.198654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb219ab3-9c96-4a70-ac18-a4b9fbb8a64d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:17.223021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:17.242838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b23f63-6dfe-499f-a82a-bb640806e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:17.245800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23f63-6dfe-499f-a82a-bb640806e905 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:17.245800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23f63-6dfe-499f-a82a-bb640806e905 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:17.245800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b23f63-6dfe-499f-a82a-bb640806e905 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:17.245800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b23f63-6dfe-499f-a82a-bb640806e905 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:17.245800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b23f63-6dfe-499f-a82a-bb640806e905 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:17.246766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b23f63-6dfe-499f-a82a-bb640806e905 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:17.260748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 606/1218] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:17:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:17.267584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c27d1-f7a0-4457-b39b-db0ac59f1b53 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:17.387281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5c27d1-f7a0-4457-b39b-db0ac59f1b53 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:17.420308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:17.427743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b42cc28-f98e-4875-aa80-6cd119b56f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:17.428446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42cc28-f98e-4875-aa80-6cd119b56f4a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:17.428734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42cc28-f98e-4875-aa80-6cd119b56f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:17.429008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b42cc28-f98e-4875-aa80-6cd119b56f4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:17.429874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42cc28-f98e-4875-aa80-6cd119b56f4a None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=2e97119806774c2482a9886b11ab6f70) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:17.430186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42cc28-f98e-4875-aa80-6cd119b56f4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:17.431607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b42cc28-f98e-4875-aa80-6cd119b56f4a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:17.445412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 613/1219] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:17:17 2022] GET /identity/v3/role_assignments?user.id=2e97119806774c2482a9886b11ab6f70&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:19.497506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca3b87-091c-4707-ab32-69f80c16eec3 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:19.498159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca3b87-091c-4707-ab32-69f80c16eec3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:19.498159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca3b87-091c-4707-ab32-69f80c16eec3 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:19.499506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 607/1220] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:19.505164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccec89-fe7d-4fca-ada1-46c92c7d9051 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:19.505647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccec89-fe7d-4fca-ada1-46c92c7d9051 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:19.505933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccec89-fe7d-4fca-ada1-46c92c7d9051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:19.524347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-27ccec89-fe7d-4fca-ada1-46c92c7d9051 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:19.525805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-27ccec89-fe7d-4fca-ada1-46c92c7d9051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:19.597643 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 614/1221] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:19.621026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca3b87-091c-4707-ab32-69f80c16eec3 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:19.733187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca3b87-091c-4707-ab32-69f80c16eec3 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:19.771009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210d374-c515-493f-8c61-5adad15879ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:19.771786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210d374-c515-493f-8c61-5adad15879ef None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:19.772076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210d374-c515-493f-8c61-5adad15879ef None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:19.772355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3210d374-c515-493f-8c61-5adad15879ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:19.783007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3210d374-c515-493f-8c61-5adad15879ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3210d374-c515-493f-8c61-5adad15879ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:17:19.790944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:19.792043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:19.792937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 608/1222] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:17:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:19.799147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccec89-fe7d-4fca-ada1-46c92c7d9051 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:19.903538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccec89-fe7d-4fca-ada1-46c92c7d9051 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:19.930203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:19.942923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da9dff-8b0e-426b-b220-21c3681ba392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:19.943599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da9dff-8b0e-426b-b220-21c3681ba392 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:19.943883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da9dff-8b0e-426b-b220-21c3681ba392 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:19.944159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84da9dff-8b0e-426b-b220-21c3681ba392 None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:19.944907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da9dff-8b0e-426b-b220-21c3681ba392 None admin] RBAC: Adding query filter params (name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:19.945226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da9dff-8b0e-426b-b220-21c3681ba392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:19.946622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84da9dff-8b0e-426b-b220-21c3681ba392 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:19.955693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 615/1223] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:17:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:19.961385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210d374-c515-493f-8c61-5adad15879ef None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:20.005669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:20.066644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3210d374-c515-493f-8c61-5adad15879ef None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:20.105762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f8e6d-6bb8-45ac-837d-4179a3fb1d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:20.106778 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f8e6d-6bb8-45ac-837d-4179a3fb1d98 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:20.107056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f8e6d-6bb8-45ac-837d-4179a3fb1d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:20.107327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f8e6d-6bb8-45ac-837d-4179a3fb1d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:20.121666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f8e6d-6bb8-45ac-837d-4179a3fb1d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:20.123271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92f8e6d-6bb8-45ac-837d-4179a3fb1d98 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-a92f8e6d-6bb8-45ac-837d-4179a3fb1d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:20.146150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:20.147138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:20.148035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:20.148035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:20.148035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:20.148035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:20.148035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 609/1224] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:20.154935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da9dff-8b0e-426b-b220-21c3681ba392 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:20.269464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84da9dff-8b0e-426b-b220-21c3681ba392 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:20.306272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8b328-dc91-4544-98aa-4c6a549f0e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:20.306937 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8b328-dc91-4544-98aa-4c6a549f0e51 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:20.307225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8b328-dc91-4544-98aa-4c6a549f0e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:20.307505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8b328-dc91-4544-98aa-4c6a549f0e51 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:20.308250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8b328-dc91-4544-98aa-4c6a549f0e51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:20.308538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8b328-dc91-4544-98aa-4c6a549f0e51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:20.310673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec8b328-dc91-4544-98aa-4c6a549f0e51 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:20.322718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 616/1225] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:20.329527 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f8e6d-6bb8-45ac-837d-4179a3fb1d98 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:20.432506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:20.441866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92f8e6d-6bb8-45ac-837d-4179a3fb1d98 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:20.478805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999869f-b2b2-49c5-969c-ba3ba6574f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:20.479647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999869f-b2b2-49c5-969c-ba3ba6574f6d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:20.479959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999869f-b2b2-49c5-969c-ba3ba6574f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:20.480276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2999869f-b2b2-49c5-969c-ba3ba6574f6d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:20.489497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999869f-b2b2-49c5-969c-ba3ba6574f6d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:20.491225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2999869f-b2b2-49c5-969c-ba3ba6574f6d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-2999869f-b2b2-49c5-969c-ba3ba6574f6d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:17:20.507357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:20.509143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:20.510372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 610/1226] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:20.515643 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8b328-dc91-4544-98aa-4c6a549f0e51 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:20.570051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:20.626846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8b328-dc91-4544-98aa-4c6a549f0e51 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:20.663490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ec708-7853-4ad3-9f2a-5ce79014dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:20.664266 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ec708-7853-4ad3-9f2a-5ce79014dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:20.664655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ec708-7853-4ad3-9f2a-5ce79014dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:20.665003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ec708-7853-4ad3-9f2a-5ce79014dfa6 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:20.666107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ec708-7853-4ad3-9f2a-5ce79014dfa6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:20.666600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ec708-7853-4ad3-9f2a-5ce79014dfa6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:20.668327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ec708-7853-4ad3-9f2a-5ce79014dfa6 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:20.690546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 617/1227] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 12 13:17:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:20.696612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999869f-b2b2-49c5-969c-ba3ba6574f6d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:20.803214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2999869f-b2b2-49c5-969c-ba3ba6574f6d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:20.841699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f2a96-186a-4551-90cf-7def7fd58a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:20.842193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f2a96-186a-4551-90cf-7def7fd58a63 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:20.842473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f2a96-186a-4551-90cf-7def7fd58a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:20.842754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609f2a96-186a-4551-90cf-7def7fd58a63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:20.856145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f2a96-186a-4551-90cf-7def7fd58a63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:20.857734 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609f2a96-186a-4551-90cf-7def7fd58a63 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-609f2a96-186a-4551-90cf-7def7fd58a63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:20.876016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:20.878084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:20.879206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:20.879206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:20.879206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:20.879206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:20.879206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 611/1228] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:20.883184 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ec708-7853-4ad3-9f2a-5ce79014dfa6 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:21.004395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ec708-7853-4ad3-9f2a-5ce79014dfa6 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:21.006700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:21.045488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e20fb9-2f19-4c5d-926f-cc979e369f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:21.046238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e20fb9-2f19-4c5d-926f-cc979e369f31 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:21.046526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e20fb9-2f19-4c5d-926f-cc979e369f31 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:21.046807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e20fb9-2f19-4c5d-926f-cc979e369f31 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:21.047537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e20fb9-2f19-4c5d-926f-cc979e369f31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:21.047848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e20fb9-2f19-4c5d-926f-cc979e369f31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:21.049217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e20fb9-2f19-4c5d-926f-cc979e369f31 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:21.061885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 618/1229] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:17:21.067956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f2a96-186a-4551-90cf-7def7fd58a63 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:21.078331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:21.188089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609f2a96-186a-4551-90cf-7def7fd58a63 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:21.226939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bed258-d224-4fb3-a604-15b77b080706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:21.227928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bed258-d224-4fb3-a604-15b77b080706 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:21.228209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bed258-d224-4fb3-a604-15b77b080706 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:21.228451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bed258-d224-4fb3-a604-15b77b080706 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:21.236429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bed258-d224-4fb3-a604-15b77b080706 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:21.237960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bed258-d224-4fb3-a604-15b77b080706 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-82bed258-d224-4fb3-a604-15b77b080706 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:17:21.254504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:21.255810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:21.257057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:17:21.257057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:17:21.257057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:17:21.257057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:17:21.257057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:17:21.257057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:17:21.257057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:21.257057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:21.257057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 612/1230] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:17:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:21.261812 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e20fb9-2f19-4c5d-926f-cc979e369f31 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:21.367172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e20fb9-2f19-4c5d-926f-cc979e369f31 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:21.403193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da28bc-f7a8-4e31-85a2-87c22586fe66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:21.403847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da28bc-f7a8-4e31-85a2-87c22586fe66 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:21.404133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da28bc-f7a8-4e31-85a2-87c22586fe66 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:21.404409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da28bc-f7a8-4e31-85a2-87c22586fe66 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:21.405199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da28bc-f7a8-4e31-85a2-87c22586fe66 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:21.405545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da28bc-f7a8-4e31-85a2-87c22586fe66 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:21.406978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da28bc-f7a8-4e31-85a2-87c22586fe66 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:21.420605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 619/1231] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:17:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:21.426720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bed258-d224-4fb3-a604-15b77b080706 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:21.494564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:21.529548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bed258-d224-4fb3-a604-15b77b080706 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:21.567041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fb3c93-f37b-47e9-b98e-2cf45f5d9ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:21.568048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb3c93-f37b-47e9-b98e-2cf45f5d9ac7 None admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:21.568342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb3c93-f37b-47e9-b98e-2cf45f5d9ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:21.568623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fb3c93-f37b-47e9-b98e-2cf45f5d9ac7 None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/2e97119806774c2482a9886b11ab6f70/roles/bd68bb224f0b4111bf407cf17e7143a4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:21.585089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb3c93-f37b-47e9-b98e-2cf45f5d9ac7 None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=2e97119806774c2482a9886b11ab6f70, role_id=bd68bb224f0b4111bf407cf17e7143a4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:21.586584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fb3c93-f37b-47e9-b98e-2cf45f5d9ac7 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:21.599514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 613/1232] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:17:21 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/2e97119806774c2482a9886b11ab6f70/roles/bd68bb224f0b4111bf407cf17e7143a4 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:17:23.753069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bae75-d6cd-4f29-b263-980eed7d7334 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:23.753683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bae75-d6cd-4f29-b263-980eed7d7334 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:23.753683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bae75-d6cd-4f29-b263-980eed7d7334 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:23.755329 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 620/1233] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:23.760650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6556a17-64ce-4c27-b326-54cee8ec104c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:23.761097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6556a17-64ce-4c27-b326-54cee8ec104c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:23.761206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6556a17-64ce-4c27-b326-54cee8ec104c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:23.777607 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f6556a17-64ce-4c27-b326-54cee8ec104c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:23.778994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f6556a17-64ce-4c27-b326-54cee8ec104c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:23.849012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 614/1234] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:23.874765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bae75-d6cd-4f29-b263-980eed7d7334 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:23.902649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:23.997472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bae75-d6cd-4f29-b263-980eed7d7334 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:24.037927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d06626-064d-4251-8ab3-9306b9249385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:24.038647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06626-064d-4251-8ab3-9306b9249385 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:24.038936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06626-064d-4251-8ab3-9306b9249385 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:24.039210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d06626-064d-4251-8ab3-9306b9249385 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:24.050673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d06626-064d-4251-8ab3-9306b9249385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-f6d06626-064d-4251-8ab3-9306b9249385 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:17:24.058537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:24.060133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:24.061016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 621/1235] 10.222.0.23 () {56 vars in 1102 bytes} [Tue Jul 12 13:17:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:24.066567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6556a17-64ce-4c27-b326-54cee8ec104c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:24.121976 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:24.187130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6556a17-64ce-4c27-b326-54cee8ec104c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:24.228074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c700ab-7933-441a-a8a6-0a909b04c46f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:24.228797 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c700ab-7933-441a-a8a6-0a909b04c46f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:24.229084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c700ab-7933-441a-a8a6-0a909b04c46f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:24.229367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c700ab-7933-441a-a8a6-0a909b04c46f None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:24.230281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c700ab-7933-441a-a8a6-0a909b04c46f None admin] RBAC: Adding query filter params (name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:24.230652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c700ab-7933-441a-a8a6-0a909b04c46f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:24.233022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c700ab-7933-441a-a8a6-0a909b04c46f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:24.242985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 615/1236] 10.222.0.23 () {56 vars in 1103 bytes} [Tue Jul 12 13:17:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:24.248850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d06626-064d-4251-8ab3-9306b9249385 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:24.346535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:24.365454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d06626-064d-4251-8ab3-9306b9249385 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:24.402534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51e5c6-22dd-4ae3-ac5b-008a1d3551f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:24.403365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51e5c6-22dd-4ae3-ac5b-008a1d3551f7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:24.403656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51e5c6-22dd-4ae3-ac5b-008a1d3551f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:24.403936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51e5c6-22dd-4ae3-ac5b-008a1d3551f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:24.414592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51e5c6-22dd-4ae3-ac5b-008a1d3551f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:24.416057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a51e5c6-22dd-4ae3-ac5b-008a1d3551f7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-3a51e5c6-22dd-4ae3-ac5b-008a1d3551f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:24.433925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:24.436145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:24.437511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:24.437511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:24.437511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:24.437511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:24.442280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c700ab-7933-441a-a8a6-0a909b04c46f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:24.442712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 622/1237] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:24.546720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c700ab-7933-441a-a8a6-0a909b04c46f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:24.557022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:24.584779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f23443-5ade-4af0-806d-6c4956ebfc4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:24.585519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f23443-5ade-4af0-806d-6c4956ebfc4d None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:24.585806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f23443-5ade-4af0-806d-6c4956ebfc4d None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:24.586125 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f23443-5ade-4af0-806d-6c4956ebfc4d None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:24.586917 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f23443-5ade-4af0-806d-6c4956ebfc4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:24.587225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f23443-5ade-4af0-806d-6c4956ebfc4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:24.588716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f23443-5ade-4af0-806d-6c4956ebfc4d None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:24.600578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 616/1238] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:17:24.606860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51e5c6-22dd-4ae3-ac5b-008a1d3551f7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:24.713760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a51e5c6-22dd-4ae3-ac5b-008a1d3551f7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:24.748909 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc797c-b94d-4fb3-943d-8331faceab28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:24.749754 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc797c-b94d-4fb3-943d-8331faceab28 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:24.750039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc797c-b94d-4fb3-943d-8331faceab28 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:24.750317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fc797c-b94d-4fb3-943d-8331faceab28 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:24.753063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:24.758317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc797c-b94d-4fb3-943d-8331faceab28 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:24.759952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fc797c-b94d-4fb3-943d-8331faceab28 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-13fc797c-b94d-4fb3-943d-8331faceab28 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:17:24.774469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:24.775627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:24.776891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 623/1239] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:24.782157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f23443-5ade-4af0-806d-6c4956ebfc4d None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:24.888409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f23443-5ade-4af0-806d-6c4956ebfc4d None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:24.925064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4b20e-99b7-4feb-8fba-46509d1aa640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:24.925796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4b20e-99b7-4feb-8fba-46509d1aa640 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:24.926093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4b20e-99b7-4feb-8fba-46509d1aa640 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:24.926386 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4b20e-99b7-4feb-8fba-46509d1aa640 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:24.927268 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4b20e-99b7-4feb-8fba-46509d1aa640 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:24.927561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4b20e-99b7-4feb-8fba-46509d1aa640 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:24.929014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4b20e-99b7-4feb-8fba-46509d1aa640 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:24.950951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 617/1240] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 12 13:17:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:24.958428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc797c-b94d-4fb3-943d-8331faceab28 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:25.066119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fc797c-b94d-4fb3-943d-8331faceab28 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:25.108112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5d53-e6d3-426e-b73c-39c5c21d4c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:25.108928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5d53-e6d3-426e-b73c-39c5c21d4c54 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:25.109181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5d53-e6d3-426e-b73c-39c5c21d4c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:25.109593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f5d53-e6d3-426e-b73c-39c5c21d4c54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:25.182010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:25.190838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f5d53-e6d3-426e-b73c-39c5c21d4c54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:25.192316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f5d53-e6d3-426e-b73c-39c5c21d4c54 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:25.309871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 624/1241] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:17:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 12 13:17:25.313981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4b20e-99b7-4feb-8fba-46509d1aa640 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:25.342879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:25.423322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4b20e-99b7-4feb-8fba-46509d1aa640 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:25.459613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c6df3-39b0-4007-bf2b-e3de8252b72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:25.460415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c6df3-39b0-4007-bf2b-e3de8252b72f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:25.460699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c6df3-39b0-4007-bf2b-e3de8252b72f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:25.460984 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5c6df3-39b0-4007-bf2b-e3de8252b72f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:25.471586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c6df3-39b0-4007-bf2b-e3de8252b72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:25.473032 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5c6df3-39b0-4007-bf2b-e3de8252b72f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-7d5c6df3-39b0-4007-bf2b-e3de8252b72f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:25.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:25.492816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:25.494154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:25.494154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:25.494154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:25.494154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:25.494154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 618/1242] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:25.500209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5d53-e6d3-426e-b73c-39c5c21d4c54 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:25.606307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:25.612960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f5d53-e6d3-426e-b73c-39c5c21d4c54 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:25.651709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b48e6-73c9-478b-aabe-41617b7f513c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:25.652317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b48e6-73c9-478b-aabe-41617b7f513c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:25.652606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b48e6-73c9-478b-aabe-41617b7f513c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:25.652951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b48e6-73c9-478b-aabe-41617b7f513c None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:25.653931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b48e6-73c9-478b-aabe-41617b7f513c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:25.654248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b48e6-73c9-478b-aabe-41617b7f513c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:25.655770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b48e6-73c9-478b-aabe-41617b7f513c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:25.672863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 625/1243] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:25.674050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c6df3-39b0-4007-bf2b-e3de8252b72f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:25.773264 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:25.796819 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5c6df3-39b0-4007-bf2b-e3de8252b72f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:25.835015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f350642-d5cd-46c8-9557-95a331d46a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:25.835807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f350642-d5cd-46c8-9557-95a331d46a37 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:25.836089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f350642-d5cd-46c8-9557-95a331d46a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:25.836363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f350642-d5cd-46c8-9557-95a331d46a37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:25.843128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f350642-d5cd-46c8-9557-95a331d46a37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:25.844564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f350642-d5cd-46c8-9557-95a331d46a37 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-2f350642-d5cd-46c8-9557-95a331d46a37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:17:25.859787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:25.860925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:25.861957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:17:25.861957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:17:25.861957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:17:25.861957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:17:25.861957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:17:25.861957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:17:25.861957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:25.861957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:25.861957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 619/1244] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:17:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:17:25.867349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b48e6-73c9-478b-aabe-41617b7f513c None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:25.978550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b48e6-73c9-478b-aabe-41617b7f513c None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:26.012732 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7e3bbb-7f09-4711-b002-1a812c4e3030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:26.013103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3bbb-7f09-4711-b002-1a812c4e3030 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:26.013412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3bbb-7f09-4711-b002-1a812c4e3030 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:26.013686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7e3bbb-7f09-4711-b002-1a812c4e3030 None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:26.014525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3bbb-7f09-4711-b002-1a812c4e3030 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:26.014840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3bbb-7f09-4711-b002-1a812c4e3030 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:26.016265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7e3bbb-7f09-4711-b002-1a812c4e3030 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:26.028336 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 626/1245] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:17:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:26.035088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f350642-d5cd-46c8-9557-95a331d46a37 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:26.152173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f350642-d5cd-46c8-9557-95a331d46a37 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:26.193920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e4f077-8344-4588-9343-6e858b7a3c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:26.194898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f077-8344-4588-9343-6e858b7a3c5a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:26.195278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f077-8344-4588-9343-6e858b7a3c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:26.195702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e4f077-8344-4588-9343-6e858b7a3c5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:26.196849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4f077-8344-4588-9343-6e858b7a3c5a None admin] RBAC: Adding query filter params (role.id=bd68bb224f0b4111bf407cf17e7143a4, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=2e97119806774c2482a9886b11ab6f70) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:26.197238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4f077-8344-4588-9343-6e858b7a3c5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:26.199020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e4f077-8344-4588-9343-6e858b7a3c5a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:26.206721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 620/1246] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:17:26 2022] GET /identity/v3/role_assignments?user.id=2e97119806774c2482a9886b11ab6f70&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=bd68bb224f0b4111bf407cf17e7143a4 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:28.468573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5b5ad-a7b0-482b-9532-8432fee58c70 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:28.469122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5b5ad-a7b0-482b-9532-8432fee58c70 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:28.469122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa5b5ad-a7b0-482b-9532-8432fee58c70 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:28.470573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 627/1247] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:28.477404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a7f9b-8724-40c2-a875-057c7b4d6147 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:28.477404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a7f9b-8724-40c2-a875-057c7b4d6147 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:28.477404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a7f9b-8724-40c2-a875-057c7b4d6147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:28.483628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:28.498538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-777a7f9b-8724-40c2-a875-057c7b4d6147 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:28.499719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-777a7f9b-8724-40c2-a875-057c7b4d6147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:28.574603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 621/1248] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:28.598665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5b5ad-a7b0-482b-9532-8432fee58c70 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:28.617351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:28.711865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa5b5ad-a7b0-482b-9532-8432fee58c70 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:28.755068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f669-de9e-48f9-9866-9e77bd4f1b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:28.755930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f669-de9e-48f9-9866-9e77bd4f1b08 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:28.756346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f669-de9e-48f9-9866-9e77bd4f1b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:28.756717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6f669-de9e-48f9-9866-9e77bd4f1b08 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:28.769512 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6f669-de9e-48f9-9866-9e77bd4f1b08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-7de6f669-de9e-48f9-9866-9e77bd4f1b08 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:17:28.778485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:28.779573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:28.780448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 628/1249] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:17:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:28.787125 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a7f9b-8724-40c2-a875-057c7b4d6147 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:28.923284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a7f9b-8724-40c2-a875-057c7b4d6147 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:28.965190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:28.972251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b111e-f4dc-49bf-8828-4b881fb6a389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:28.973076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b111e-f4dc-49bf-8828-4b881fb6a389 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:28.973470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b111e-f4dc-49bf-8828-4b881fb6a389 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:28.973852 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6b111e-f4dc-49bf-8828-4b881fb6a389 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:28.974801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b111e-f4dc-49bf-8828-4b881fb6a389 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:28.975203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b111e-f4dc-49bf-8828-4b881fb6a389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:28.977023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6b111e-f4dc-49bf-8828-4b881fb6a389 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:28.986999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 622/1250] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:17:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:28.993033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f669-de9e-48f9-9866-9e77bd4f1b08 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:29.083670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:29.110835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6f669-de9e-48f9-9866-9e77bd4f1b08 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:29.150634 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17f006-0338-4ca0-a1a0-d605f03f7409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:29.151472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17f006-0338-4ca0-a1a0-d605f03f7409 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:29.151756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17f006-0338-4ca0-a1a0-d605f03f7409 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:29.152030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb17f006-0338-4ca0-a1a0-d605f03f7409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:29.167256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17f006-0338-4ca0-a1a0-d605f03f7409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:29.169655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb17f006-0338-4ca0-a1a0-d605f03f7409 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-fb17f006-0338-4ca0-a1a0-d605f03f7409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:29.191406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:29.192706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:29.193773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:29.193773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:29.193773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:29.193773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:29.193773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 629/1251] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:29.200173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b111e-f4dc-49bf-8828-4b881fb6a389 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:29.319191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6b111e-f4dc-49bf-8828-4b881fb6a389 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:29.359413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0503cfe-eb54-4547-957e-f1aa8875c935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:29.360337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0503cfe-eb54-4547-957e-f1aa8875c935 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:29.360665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0503cfe-eb54-4547-957e-f1aa8875c935 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:29.360983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0503cfe-eb54-4547-957e-f1aa8875c935 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:29.361974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0503cfe-eb54-4547-957e-f1aa8875c935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:29.362274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0503cfe-eb54-4547-957e-f1aa8875c935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:29.363971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0503cfe-eb54-4547-957e-f1aa8875c935 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:29.378129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 623/1252] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:29.384969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17f006-0338-4ca0-a1a0-d605f03f7409 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:29.493059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb17f006-0338-4ca0-a1a0-d605f03f7409 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:29.522845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:29.532773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4cf63-24ab-4c7f-a421-0a2e6a16e88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:29.533661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4cf63-24ab-4c7f-a421-0a2e6a16e88d None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:29.533932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4cf63-24ab-4c7f-a421-0a2e6a16e88d None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:29.534212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd4cf63-24ab-4c7f-a421-0a2e6a16e88d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:29.543153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4cf63-24ab-4c7f-a421-0a2e6a16e88d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:29.545318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd4cf63-24ab-4c7f-a421-0a2e6a16e88d None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-bdd4cf63-24ab-4c7f-a421-0a2e6a16e88d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:17:29.560525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:29.568651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:29.570130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 630/1253] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:29.571015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0503cfe-eb54-4547-957e-f1aa8875c935 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:29.635232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:29.687653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0503cfe-eb54-4547-957e-f1aa8875c935 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:29.726270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b42bcd-2eb2-422f-a326-537930ec611a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:29.727047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b42bcd-2eb2-422f-a326-537930ec611a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:29.727397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b42bcd-2eb2-422f-a326-537930ec611a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:29.727670 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b42bcd-2eb2-422f-a326-537930ec611a None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:29.728568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b42bcd-2eb2-422f-a326-537930ec611a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:29.728927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b42bcd-2eb2-422f-a326-537930ec611a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:29.730507 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b42bcd-2eb2-422f-a326-537930ec611a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:29.751279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 624/1254] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 12 13:17:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:29.756814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4cf63-24ab-4c7f-a421-0a2e6a16e88d None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:29.871359 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd4cf63-24ab-4c7f-a421-0a2e6a16e88d None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:29.913591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8a7fe-44e3-4a08-b4d6-a136ecfe41d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:29.914397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8a7fe-44e3-4a08-b4d6-a136ecfe41d7 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:29.914708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8a7fe-44e3-4a08-b4d6-a136ecfe41d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:29.915129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf8a7fe-44e3-4a08-b4d6-a136ecfe41d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:29.992435 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:30.006915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8a7fe-44e3-4a08-b4d6-a136ecfe41d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:30.008504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf8a7fe-44e3-4a08-b4d6-a136ecfe41d7 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:30.122972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 631/1255] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:17:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:17:30.124782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b42bcd-2eb2-422f-a326-537930ec611a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:30.240925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b42bcd-2eb2-422f-a326-537930ec611a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:30.258165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:30.283968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a33d1-49b2-4ce7-8519-ac0e78c1cea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:30.284843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a33d1-49b2-4ce7-8519-ac0e78c1cea9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:30.285122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a33d1-49b2-4ce7-8519-ac0e78c1cea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:30.285440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026a33d1-49b2-4ce7-8519-ac0e78c1cea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:30.297980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a33d1-49b2-4ce7-8519-ac0e78c1cea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:30.299608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026a33d1-49b2-4ce7-8519-ac0e78c1cea9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-026a33d1-49b2-4ce7-8519-ac0e78c1cea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:30.319928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:30.321083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:30.322274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:30.322274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:30.322274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:30.322274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:30.322274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 625/1256] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:30.332013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8a7fe-44e3-4a08-b4d6-a136ecfe41d7 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:30.430072 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:30.444780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf8a7fe-44e3-4a08-b4d6-a136ecfe41d7 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:30.482841 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26688b-02ae-40c2-84d2-6f35691b3227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:30.483509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26688b-02ae-40c2-84d2-6f35691b3227 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:30.483793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26688b-02ae-40c2-84d2-6f35691b3227 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:30.484071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d26688b-02ae-40c2-84d2-6f35691b3227 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:30.484831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26688b-02ae-40c2-84d2-6f35691b3227 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:30.485152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26688b-02ae-40c2-84d2-6f35691b3227 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:30.486678 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d26688b-02ae-40c2-84d2-6f35691b3227 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:30.499258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 632/1257] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:30.505414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a33d1-49b2-4ce7-8519-ac0e78c1cea9 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:30.613188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026a33d1-49b2-4ce7-8519-ac0e78c1cea9 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:30.649253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f7de6-34bd-4bad-87ee-72e633e23a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:30.650114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f7de6-34bd-4bad-87ee-72e633e23a92 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:30.650398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f7de6-34bd-4bad-87ee-72e633e23a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:30.650687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42f7de6-34bd-4bad-87ee-72e633e23a92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:30.657518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f7de6-34bd-4bad-87ee-72e633e23a92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:30.658997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42f7de6-34bd-4bad-87ee-72e633e23a92 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-c42f7de6-34bd-4bad-87ee-72e633e23a92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:17:30.674753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:30.675879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:30.677139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:17:30.677139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:17:30.677139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:17:30.677139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:17:30.677139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:17:30.677139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:17:30.677139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:30.677139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:30.677139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 626/1258] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:17:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 12 13:17:30.682081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26688b-02ae-40c2-84d2-6f35691b3227 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:30.799871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d26688b-02ae-40c2-84d2-6f35691b3227 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:30.835109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4877be1-84b5-4942-9c9b-ac8077b6e82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:30.835777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877be1-84b5-4942-9c9b-ac8077b6e82f None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:30.836061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877be1-84b5-4942-9c9b-ac8077b6e82f None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:30.836337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4877be1-84b5-4942-9c9b-ac8077b6e82f None admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:30.837133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4877be1-84b5-4942-9c9b-ac8077b6e82f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:30.837475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4877be1-84b5-4942-9c9b-ac8077b6e82f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:30.838914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4877be1-84b5-4942-9c9b-ac8077b6e82f None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:30.852016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 633/1259] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:17:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:30.859474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f7de6-34bd-4bad-87ee-72e633e23a92 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:30.889518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:30.979388 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42f7de6-34bd-4bad-87ee-72e633e23a92 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:31.025315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe2d433-3f18-4561-8212-fe088165eb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:31.026022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2d433-3f18-4561-8212-fe088165eb31 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:31.026309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2d433-3f18-4561-8212-fe088165eb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:31.026583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe2d433-3f18-4561-8212-fe088165eb31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:31.027435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2d433-3f18-4561-8212-fe088165eb31 None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=2e97119806774c2482a9886b11ab6f70) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:31.028306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2d433-3f18-4561-8212-fe088165eb31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:31.029819 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe2d433-3f18-4561-8212-fe088165eb31 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:31.036076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 627/1260] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:17:30 2022] GET /identity/v3/role_assignments?user.id=2e97119806774c2482a9886b11ab6f70&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:33.054124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf516a-32cd-4123-860b-738d6cd24a70 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:33.054682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf516a-32cd-4123-860b-738d6cd24a70 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:33.054682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cf516a-32cd-4123-860b-738d6cd24a70 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:33.056088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 634/1261] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:33.061807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185762-7289-4961-8d9d-4d74571c02d9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:33.062190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185762-7289-4961-8d9d-4d74571c02d9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:33.062392 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64185762-7289-4961-8d9d-4d74571c02d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:33.079706 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-64185762-7289-4961-8d9d-4d74571c02d9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:33.080882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-64185762-7289-4961-8d9d-4d74571c02d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:33.144925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 628/1262] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:33.166272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf516a-32cd-4123-860b-738d6cd24a70 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:33.176864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:33.265537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cf516a-32cd-4123-860b-738d6cd24a70 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:33.297722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9254409-5a90-4cd3-b161-5e2e242af62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:33.298503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9254409-5a90-4cd3-b161-5e2e242af62b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:33.298784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9254409-5a90-4cd3-b161-5e2e242af62b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:33.299057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9254409-5a90-4cd3-b161-5e2e242af62b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:33.308700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9254409-5a90-4cd3-b161-5e2e242af62b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e9254409-5a90-4cd3-b161-5e2e242af62b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:17:33.317172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:33.318289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:33.319166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 635/1263] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:17:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:33.324536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64185762-7289-4961-8d9d-4d74571c02d9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:33.344201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:33.422887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64185762-7289-4961-8d9d-4d74571c02d9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:33.455495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a7b77-816d-4532-9ffd-c92d1fe2f5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:33.456150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a7b77-816d-4532-9ffd-c92d1fe2f5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:33.456433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a7b77-816d-4532-9ffd-c92d1fe2f5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:33.456707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a7b77-816d-4532-9ffd-c92d1fe2f5c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:33.457476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a7b77-816d-4532-9ffd-c92d1fe2f5c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:33.457782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a7b77-816d-4532-9ffd-c92d1fe2f5c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:33.459130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936a7b77-816d-4532-9ffd-c92d1fe2f5c0 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:33.466972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 629/1264] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:17:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:33.472773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9254409-5a90-4cd3-b161-5e2e242af62b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:33.537456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:33.570439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9254409-5a90-4cd3-b161-5e2e242af62b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:33.603132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208020a-8291-4cbe-a891-6d3c5d07ef1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:33.603914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208020a-8291-4cbe-a891-6d3c5d07ef1b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:33.604196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208020a-8291-4cbe-a891-6d3c5d07ef1b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:33.604477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208020a-8291-4cbe-a891-6d3c5d07ef1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:33.614783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208020a-8291-4cbe-a891-6d3c5d07ef1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:33.616226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208020a-8291-4cbe-a891-6d3c5d07ef1b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-e208020a-8291-4cbe-a891-6d3c5d07ef1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:33.632736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:33.633810 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:33.634876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:33.634876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:33.634876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:33.634876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:33.634876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 636/1265] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:33.640481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a7b77-816d-4532-9ffd-c92d1fe2f5c0 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:33.720978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:33.739540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a7b77-816d-4532-9ffd-c92d1fe2f5c0 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:33.772078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2a8a9-2719-43ad-94c1-ec5907e005f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:33.772718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2a8a9-2719-43ad-94c1-ec5907e005f2 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:33.773001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2a8a9-2719-43ad-94c1-ec5907e005f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:33.773276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c2a8a9-2719-43ad-94c1-ec5907e005f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:33.774023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2a8a9-2719-43ad-94c1-ec5907e005f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:33.774329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2a8a9-2719-43ad-94c1-ec5907e005f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:33.775680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c2a8a9-2719-43ad-94c1-ec5907e005f2 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:33.786883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 630/1266] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:33.792533 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208020a-8291-4cbe-a891-6d3c5d07ef1b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:33.889756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208020a-8291-4cbe-a891-6d3c5d07ef1b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:33.910436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:33.926492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47e9df-3582-4eba-85be-a991dbb9feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:33.927302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47e9df-3582-4eba-85be-a991dbb9feed None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:33.927575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47e9df-3582-4eba-85be-a991dbb9feed None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:33.927863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b47e9df-3582-4eba-85be-a991dbb9feed None admin] PATH_INFO: `/v3/users/placement` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:33.936217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47e9df-3582-4eba-85be-a991dbb9feed None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:33.937655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b47e9df-3582-4eba-85be-a991dbb9feed None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-6b47e9df-3582-4eba-85be-a991dbb9feed None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:17:33.952702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:33.953919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:33.954931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 637/1267] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:33.960255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2a8a9-2719-43ad-94c1-ec5907e005f2 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:34.059642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c2a8a9-2719-43ad-94c1-ec5907e005f2 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:34.095498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45c5cd-c550-4938-8644-809107c27c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:34.096161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45c5cd-c550-4938-8644-809107c27c15 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:34.096445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45c5cd-c550-4938-8644-809107c27c15 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:34.096722 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f45c5cd-c550-4938-8644-809107c27c15 None admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:34.097589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45c5cd-c550-4938-8644-809107c27c15 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:34.097900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45c5cd-c550-4938-8644-809107c27c15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:34.099320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f45c5cd-c550-4938-8644-809107c27c15 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:34.108340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:34.116962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 631/1268] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 12 13:17:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:34.123055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47e9df-3582-4eba-85be-a991dbb9feed None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:34.224167 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b47e9df-3582-4eba-85be-a991dbb9feed None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:34.257119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468be44-6fc6-4bad-a79d-bffe5b8675ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:34.257936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468be44-6fc6-4bad-a79d-bffe5b8675ab None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:34.258218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468be44-6fc6-4bad-a79d-bffe5b8675ab None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:34.258492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468be44-6fc6-4bad-a79d-bffe5b8675ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:34.269110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468be44-6fc6-4bad-a79d-bffe5b8675ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:34.270581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2468be44-6fc6-4bad-a79d-bffe5b8675ab None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-2468be44-6fc6-4bad-a79d-bffe5b8675ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:34.287538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:34.288581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:34.289708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:34.289708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:34.289708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:34.289708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:34.289708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 638/1269] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:34.294359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45c5cd-c550-4938-8644-809107c27c15 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:34.390377 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f45c5cd-c550-4938-8644-809107c27c15 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:34.423064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b00ad3-8766-4191-b303-56492189d9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:34.423713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b00ad3-8766-4191-b303-56492189d9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:34.424000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b00ad3-8766-4191-b303-56492189d9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:34.424277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b00ad3-8766-4191-b303-56492189d9d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:34.425002 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b00ad3-8766-4191-b303-56492189d9d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:34.425310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b00ad3-8766-4191-b303-56492189d9d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:34.426702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b00ad3-8766-4191-b303-56492189d9d8 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:34.437615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 632/1270] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:34.443393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468be44-6fc6-4bad-a79d-bffe5b8675ab None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:34.475451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:34.541891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2468be44-6fc6-4bad-a79d-bffe5b8675ab None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:34.574437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf8d20-ade6-4d03-9552-d112a6f30a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:34.575866 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf8d20-ade6-4d03-9552-d112a6f30a44 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:34.576142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf8d20-ade6-4d03-9552-d112a6f30a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:34.576413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf8d20-ade6-4d03-9552-d112a6f30a44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:34.582808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf8d20-ade6-4d03-9552-d112a6f30a44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:34.584230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bf8d20-ade6-4d03-9552-d112a6f30a44 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-25bf8d20-ade6-4d03-9552-d112a6f30a44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:17:34.598088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:34.599118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:34.600356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:17:34.600356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:17:34.600356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:17:34.600356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:17:34.600356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:17:34.600356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:17:34.600356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:34.600356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:34.600356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 639/1271] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:17:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:34.604992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b00ad3-8766-4191-b303-56492189d9d8 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:34.661943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:34.703074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b00ad3-8766-4191-b303-56492189d9d8 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:34.734777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb9509-7393-40c8-a042-c233af0b02b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:34.735375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb9509-7393-40c8-a042-c233af0b02b9 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:34.735660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb9509-7393-40c8-a042-c233af0b02b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:34.735937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb9509-7393-40c8-a042-c233af0b02b9 None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:34.736723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb9509-7393-40c8-a042-c233af0b02b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:34.737030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb9509-7393-40c8-a042-c233af0b02b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:34.738493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb9509-7393-40c8-a042-c233af0b02b9 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:34.750602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 633/1272] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:17:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:34.756395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf8d20-ade6-4d03-9552-d112a6f30a44 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:34.844420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:34.853328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bf8d20-ade6-4d03-9552-d112a6f30a44 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:34.886490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274030f-c6d1-4854-9a9f-403310c0f48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:34.887383 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274030f-c6d1-4854-9a9f-403310c0f48c None admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:34.887663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274030f-c6d1-4854-9a9f-403310c0f48c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:34.887934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274030f-c6d1-4854-9a9f-403310c0f48c None admin] PATH_INFO: `/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/2e97119806774c2482a9886b11ab6f70/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:34.889877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274030f-c6d1-4854-9a9f-403310c0f48c None admin] RBAC: Authorizing `identity:create_grant(project_id=e60ff54d1d814805b912118b9b02cc28, user_id=2e97119806774c2482a9886b11ab6f70, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:34.891290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3274030f-c6d1-4854-9a9f-403310c0f48c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:34.903724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 640/1273] 10.222.0.23 () {58 vars in 1439 bytes} [Tue Jul 12 13:17:34 2022] PUT /identity/v3/projects/e60ff54d1d814805b912118b9b02cc28/users/2e97119806774c2482a9886b11ab6f70/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:17:36.919226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806d571-29ef-423a-b20a-9f8ff971fa66 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:36.919754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806d571-29ef-423a-b20a-9f8ff971fa66 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:36.919860 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806d571-29ef-423a-b20a-9f8ff971fa66 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:36.921157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 634/1274] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:36.926908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763cf73-3dc3-44e1-bf64-3e654241230e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:36.927296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763cf73-3dc3-44e1-bf64-3e654241230e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:36.927632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763cf73-3dc3-44e1-bf64-3e654241230e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:36.946797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4763cf73-3dc3-44e1-bf64-3e654241230e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:36.947995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4763cf73-3dc3-44e1-bf64-3e654241230e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:37.013200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 641/1275] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:37.035801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806d571-29ef-423a-b20a-9f8ff971fa66 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:37.134430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806d571-29ef-423a-b20a-9f8ff971fa66 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:37.151439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:37.168240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4845df-523c-4bdd-8d8d-aef833154ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:37.169075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4845df-523c-4bdd-8d8d-aef833154ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:37.169425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4845df-523c-4bdd-8d8d-aef833154ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:37.169657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4845df-523c-4bdd-8d8d-aef833154ad5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:37.179865 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4845df-523c-4bdd-8d8d-aef833154ad5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-ac4845df-523c-4bdd-8d8d-aef833154ad5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 12 13:17:37.187545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:37.189133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:37.190298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 635/1276] 10.222.0.23 () {56 vars in 1096 bytes} [Tue Jul 12 13:17:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:37.195136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763cf73-3dc3-44e1-bf64-3e654241230e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:37.276855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:37.296752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763cf73-3dc3-44e1-bf64-3e654241230e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:37.330823 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071fd42-9da0-4ca6-bdf9-6dcbc852457a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:37.331484 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071fd42-9da0-4ca6-bdf9-6dcbc852457a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:37.331773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071fd42-9da0-4ca6-bdf9-6dcbc852457a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:37.332053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8071fd42-9da0-4ca6-bdf9-6dcbc852457a None admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:37.332801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071fd42-9da0-4ca6-bdf9-6dcbc852457a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:37.333111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071fd42-9da0-4ca6-bdf9-6dcbc852457a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:37.334513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8071fd42-9da0-4ca6-bdf9-6dcbc852457a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:37.343253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 642/1277] 10.222.0.23 () {56 vars in 1099 bytes} [Tue Jul 12 13:17:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:37.348910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4845df-523c-4bdd-8d8d-aef833154ad5 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:37.445095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4845df-523c-4bdd-8d8d-aef833154ad5 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:37.477892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2eb80d-46eb-4e96-abf9-8b717b95a563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:37.478674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2eb80d-46eb-4e96-abf9-8b717b95a563 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:37.478959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2eb80d-46eb-4e96-abf9-8b717b95a563 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:37.479236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2eb80d-46eb-4e96-abf9-8b717b95a563 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:37.489543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2eb80d-46eb-4e96-abf9-8b717b95a563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:37.490977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2eb80d-46eb-4e96-abf9-8b717b95a563 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-8e2eb80d-46eb-4e96-abf9-8b717b95a563 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:37.508225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:37.509278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:37.510415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:37.510415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:37.510415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:37.510415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:37.510415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 636/1278] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:37.515819 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071fd42-9da0-4ca6-bdf9-6dcbc852457a None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:37.612520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8071fd42-9da0-4ca6-bdf9-6dcbc852457a None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:37.645294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6904a30-42bf-4a55-a8df-c33fe6f33f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:37.645968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6904a30-42bf-4a55-a8df-c33fe6f33f50 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:37.646254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6904a30-42bf-4a55-a8df-c33fe6f33f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:37.646529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6904a30-42bf-4a55-a8df-c33fe6f33f50 None admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:37.647246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6904a30-42bf-4a55-a8df-c33fe6f33f50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:37.647550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6904a30-42bf-4a55-a8df-c33fe6f33f50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:37.648911 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6904a30-42bf-4a55-a8df-c33fe6f33f50 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:37.660244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 643/1279] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:37.665763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2eb80d-46eb-4e96-abf9-8b717b95a563 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:37.697410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:37.765253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2eb80d-46eb-4e96-abf9-8b717b95a563 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:37.797749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c9cf9-9163-4e4b-87bf-8cdb66c09441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:37.798815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c9cf9-9163-4e4b-87bf-8cdb66c09441 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:37.799101 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c9cf9-9163-4e4b-87bf-8cdb66c09441 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:37.799379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c9cf9-9163-4e4b-87bf-8cdb66c09441 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:37.808313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c9cf9-9163-4e4b-87bf-8cdb66c09441 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:37.810549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c9cf9-9163-4e4b-87bf-8cdb66c09441 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-3b2c9cf9-9163-4e4b-87bf-8cdb66c09441 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 12 13:17:37.825749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:37.826853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:37.827862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 637/1280] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:37.832560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6904a30-42bf-4a55-a8df-c33fe6f33f50 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:37.836289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:37.933012 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6904a30-42bf-4a55-a8df-c33fe6f33f50 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:37.966435 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f8517-7495-4ab7-bd0f-9ef89747fe81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:37.967098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f8517-7495-4ab7-bd0f-9ef89747fe81 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:37.967384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f8517-7495-4ab7-bd0f-9ef89747fe81 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:37.967662 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166f8517-7495-4ab7-bd0f-9ef89747fe81 None admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:37.968504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f8517-7495-4ab7-bd0f-9ef89747fe81 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:37.968819 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f8517-7495-4ab7-bd0f-9ef89747fe81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:37.970304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166f8517-7495-4ab7-bd0f-9ef89747fe81 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:37.987168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 644/1281] 10.222.0.23 () {56 vars in 1143 bytes} [Tue Jul 12 13:17:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:37.992951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c9cf9-9163-4e4b-87bf-8cdb66c09441 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:38.080910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:38.092514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c9cf9-9163-4e4b-87bf-8cdb66c09441 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:38.128855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa34530-1faf-4488-9dab-67df26ba4f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:38.129532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa34530-1faf-4488-9dab-67df26ba4f0f None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:38.129824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa34530-1faf-4488-9dab-67df26ba4f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:38.130104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa34530-1faf-4488-9dab-67df26ba4f0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:38.195233 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa34530-1faf-4488-9dab-67df26ba4f0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:38.196674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa34530-1faf-4488-9dab-67df26ba4f0f None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:38.290005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:38.301443 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 638/1282] 10.222.0.23 () {58 vars in 1282 bytes} [Tue Jul 12 13:17:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 12 13:17:38.307199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f8517-7495-4ab7-bd0f-9ef89747fe81 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:38.368505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:38.408371 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166f8517-7495-4ab7-bd0f-9ef89747fe81 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:38.442671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d67773-506d-4654-9f59-c700b454d6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:38.443457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d67773-506d-4654-9f59-c700b454d6c0 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:38.443747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d67773-506d-4654-9f59-c700b454d6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:38.444026 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d67773-506d-4654-9f59-c700b454d6c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:38.454356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d67773-506d-4654-9f59-c700b454d6c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:38.455795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d67773-506d-4654-9f59-c700b454d6c0 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-12d67773-506d-4654-9f59-c700b454d6c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 12 13:17:38.473856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:38.474991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:38.476269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 12 13:17:38.476269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 12 13:17:38.476269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 12 13:17:38.476269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:38.476269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 645/1283] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:17:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:38.480787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa34530-1faf-4488-9dab-67df26ba4f0f None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:38.576805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa34530-1faf-4488-9dab-67df26ba4f0f None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:38.609780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dfff9-c839-4c24-a254-cca178467b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:38.610432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dfff9-c839-4c24-a254-cca178467b5a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:38.610719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dfff9-c839-4c24-a254-cca178467b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:38.610998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9dfff9-c839-4c24-a254-cca178467b5a None admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:38.611719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dfff9-c839-4c24-a254-cca178467b5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:38.612028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dfff9-c839-4c24-a254-cca178467b5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:38.613407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9dfff9-c839-4c24-a254-cca178467b5a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:38.624243 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 639/1284] 10.222.0.23 () {56 vars in 1109 bytes} [Tue Jul 12 13:17:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:38.629729 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d67773-506d-4654-9f59-c700b454d6c0 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:38.725786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d67773-506d-4654-9f59-c700b454d6c0 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:38.739672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:38.759719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6168-8159-40f1-adb2-fa68195d782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:38.760502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6168-8159-40f1-adb2-fa68195d782b None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:38.760788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6168-8159-40f1-adb2-fa68195d782b None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:38.761067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ac6168-8159-40f1-adb2-fa68195d782b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:38.768268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac6168-8159-40f1-adb2-fa68195d782b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:38.769738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ac6168-8159-40f1-adb2-fa68195d782b None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application [None req-09ac6168-8159-40f1-adb2-fa68195d782b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 12 13:17:38.783959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:38.784997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:38.786298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 12 13:17:38.786298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 12 13:17:38.786298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 12 13:17:38.786298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 12 13:17:38.786298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 12 13:17:38.786298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 12 13:17:38.786298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 12 13:17:38.786298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: ERROR keystone.server.flask.application Jul 12 13:17:38.786298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 646/1285] 10.222.0.23 () {56 vars in 1111 bytes} [Tue Jul 12 13:17:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:38.790892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dfff9-c839-4c24-a254-cca178467b5a None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:38.840225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:38.889125 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9dfff9-c839-4c24-a254-cca178467b5a None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:38.926616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3df4355-7a96-4107-9714-2987326da3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:38.927317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df4355-7a96-4107-9714-2987326da3de None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:38.927765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df4355-7a96-4107-9714-2987326da3de None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:38.928147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3df4355-7a96-4107-9714-2987326da3de None admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:38.929158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df4355-7a96-4107-9714-2987326da3de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:38.929534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df4355-7a96-4107-9714-2987326da3de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:38.931633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3df4355-7a96-4107-9714-2987326da3de None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:38.944687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 640/1286] 10.222.0.23 () {56 vars in 1148 bytes} [Tue Jul 12 13:17:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:38.951246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6168-8159-40f1-adb2-fa68195d782b None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:39.048638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ac6168-8159-40f1-adb2-fa68195d782b None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:39.081653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f89634-4b9d-4632-b938-5934f8a0a41a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:39.082294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f89634-4b9d-4632-b938-5934f8a0a41a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:39.082577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f89634-4b9d-4632-b938-5934f8a0a41a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:39.082853 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f89634-4b9d-4632-b938-5934f8a0a41a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:39.083672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f89634-4b9d-4632-b938-5934f8a0a41a None admin] RBAC: Adding query filter params (role.id=8ef47d10f2ff4c33b9412cbb766cea3b, scope.project.id=e60ff54d1d814805b912118b9b02cc28, user.id=2e97119806774c2482a9886b11ab6f70) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:39.083984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f89634-4b9d-4632-b938-5934f8a0a41a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:39.085414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f89634-4b9d-4632-b938-5934f8a0a41a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:39.091815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 647/1287] 10.222.0.23 () {56 vars in 1374 bytes} [Tue Jul 12 13:17:38 2022] GET /identity/v3/role_assignments?user.id=2e97119806774c2482a9886b11ab6f70&scope.project.id=e60ff54d1d814805b912118b9b02cc28&role.id=8ef47d10f2ff4c33b9412cbb766cea3b => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:41.128699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31428bf9-b052-4bbd-84d5-271d47b69230 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:41.129230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31428bf9-b052-4bbd-84d5-271d47b69230 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:41.129230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31428bf9-b052-4bbd-84d5-271d47b69230 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:41.130659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 641/1288] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:41.135730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ab9fe-b3d5-443c-b2d8-f70011b8c3c0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:41.136123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ab9fe-b3d5-443c-b2d8-f70011b8c3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:41.136295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349ab9fe-b3d5-443c-b2d8-f70011b8c3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:41.153604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-349ab9fe-b3d5-443c-b2d8-f70011b8c3c0 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:41.154794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-349ab9fe-b3d5-443c-b2d8-f70011b8c3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:41.182645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:41.220204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 648/1289] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:41.242301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31428bf9-b052-4bbd-84d5-271d47b69230 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:41.342944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31428bf9-b052-4bbd-84d5-271d47b69230 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:41.353119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:41.376676 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03835d89-38bd-48fb-80a1-c98bbcb82300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:41.377498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03835d89-38bd-48fb-80a1-c98bbcb82300 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:41.377789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03835d89-38bd-48fb-80a1-c98bbcb82300 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:41.378068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03835d89-38bd-48fb-80a1-c98bbcb82300 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:41.378771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03835d89-38bd-48fb-80a1-c98bbcb82300 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:41.380092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03835d89-38bd-48fb-80a1-c98bbcb82300 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-03835d89-38bd-48fb-80a1-c98bbcb82300 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:41.392641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:41.398801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:41.399914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 642/1290] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:17:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:41.401790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ab9fe-b3d5-443c-b2d8-f70011b8c3c0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:41.498026 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349ab9fe-b3d5-443c-b2d8-f70011b8c3c0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:41.530192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10983db5-1c4c-42f7-9b8f-5f66c0a21c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:41.530841 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10983db5-1c4c-42f7-9b8f-5f66c0a21c6a None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:41.531123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10983db5-1c4c-42f7-9b8f-5f66c0a21c6a None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:41.531394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10983db5-1c4c-42f7-9b8f-5f66c0a21c6a None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:41.532096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10983db5-1c4c-42f7-9b8f-5f66c0a21c6a None admin] RBAC: Adding query filter params (name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:41.532402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10983db5-1c4c-42f7-9b8f-5f66c0a21c6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:41.533798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10983db5-1c4c-42f7-9b8f-5f66c0a21c6a None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:41.538459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 649/1291] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jul 12 13:17:41 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:41.544335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03835d89-38bd-48fb-80a1-c98bbcb82300 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:41.641128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03835d89-38bd-48fb-80a1-c98bbcb82300 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:41.672837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bfeef-6177-4896-a1d2-70484bfda56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:41.673509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bfeef-6177-4896-a1d2-70484bfda56a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:41.673793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bfeef-6177-4896-a1d2-70484bfda56a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:41.674072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bfeef-6177-4896-a1d2-70484bfda56a None admin] PATH_INFO: `/v3/services` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:41.674774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bfeef-6177-4896-a1d2-70484bfda56a None admin] RBAC: Adding query filter params (type=placement) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:41.675079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bfeef-6177-4896-a1d2-70484bfda56a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:41.676424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bfeef-6177-4896-a1d2-70484bfda56a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:41.680654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 643/1292] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jul 12 13:17:41 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:43.708671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e804-03f5-4b4f-ac63-0906625f6091 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:43.709216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e804-03f5-4b4f-ac63-0906625f6091 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:43.709216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b76e804-03f5-4b4f-ac63-0906625f6091 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:43.710539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 650/1293] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:43.716216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6d7be-4c13-4110-92ce-b0ba889ce7f9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:43.716650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6d7be-4c13-4110-92ce-b0ba889ce7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:43.716751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d6d7be-4c13-4110-92ce-b0ba889ce7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:43.733493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-71d6d7be-4c13-4110-92ce-b0ba889ce7f9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:43.734662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-71d6d7be-4c13-4110-92ce-b0ba889ce7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:43.762207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:43.798025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 644/1294] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:43.820778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e804-03f5-4b4f-ac63-0906625f6091 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:43.851381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:43.922278 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b76e804-03f5-4b4f-ac63-0906625f6091 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:43.959328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64f99d5-bef1-4a84-9913-417f2ce4c894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:43.959898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f99d5-bef1-4a84-9913-417f2ce4c894 None admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:43.960182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f99d5-bef1-4a84-9913-417f2ce4c894 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:43.960454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64f99d5-bef1-4a84-9913-417f2ce4c894 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:43.961180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f99d5-bef1-4a84-9913-417f2ce4c894 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:43.962645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64f99d5-bef1-4a84-9913-417f2ce4c894 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:43.976564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 651/1295] 10.222.0.23 () {60 vars in 1141 bytes} [Tue Jul 12 13:17:43 2022] POST /identity/v3/services => generated 246 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:17:45.992360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd47ff1-453b-4da6-8de8-9bc71792b931 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:45.992929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd47ff1-453b-4da6-8de8-9bc71792b931 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:45.992929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd47ff1-453b-4da6-8de8-9bc71792b931 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:45.994316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 645/1296] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:45.999872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc1a03-4b5d-4985-97c7-5fb590b11f91 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:46.000286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc1a03-4b5d-4985-97c7-5fb590b11f91 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:46.000430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcc1a03-4b5d-4985-97c7-5fb590b11f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:46.017304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bdcc1a03-4b5d-4985-97c7-5fb590b11f91 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:46.018530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bdcc1a03-4b5d-4985-97c7-5fb590b11f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:46.088056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 652/1297] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:45 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:46.111044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd47ff1-453b-4da6-8de8-9bc71792b931 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:46.209923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd47ff1-453b-4da6-8de8-9bc71792b931 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:46.243331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53e76-353c-4d0c-9b1c-48bf1dfc8d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:46.244066 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53e76-353c-4d0c-9b1c-48bf1dfc8d6b None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:46.244355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53e76-353c-4d0c-9b1c-48bf1dfc8d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:46.244635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab53e76-353c-4d0c-9b1c-48bf1dfc8d6b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:46.245294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab53e76-353c-4d0c-9b1c-48bf1dfc8d6b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:46.247283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab53e76-353c-4d0c-9b1c-48bf1dfc8d6b None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-0ab53e76-353c-4d0c-9b1c-48bf1dfc8d6b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:46.259261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:46.260531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:46.261754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 646/1298] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:17:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:46.267110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc1a03-4b5d-4985-97c7-5fb590b11f91 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:46.285995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:46.365258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcc1a03-4b5d-4985-97c7-5fb590b11f91 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:46.397776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a3c81-d398-440b-96fe-4aa28755bb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:46.398430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a3c81-d398-440b-96fe-4aa28755bb13 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:46.398715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a3c81-d398-440b-96fe-4aa28755bb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:46.398990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44a3c81-d398-440b-96fe-4aa28755bb13 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:46.399694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a3c81-d398-440b-96fe-4aa28755bb13 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:46.400003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a3c81-d398-440b-96fe-4aa28755bb13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:46.401366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44a3c81-d398-440b-96fe-4aa28755bb13 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:46.407232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 653/1299] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jul 12 13:17:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:46.413115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53e76-353c-4d0c-9b1c-48bf1dfc8d6b None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:46.443632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:46.510018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab53e76-353c-4d0c-9b1c-48bf1dfc8d6b None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:46.542855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcff061-022b-4b6e-b6e0-c1e14bb91e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:46.543547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcff061-022b-4b6e-b6e0-c1e14bb91e5a None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:46.543830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcff061-022b-4b6e-b6e0-c1e14bb91e5a None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:46.544104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcff061-022b-4b6e-b6e0-c1e14bb91e5a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:46.544892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcff061-022b-4b6e-b6e0-c1e14bb91e5a None admin] RBAC: Adding query filter params (interface=public, service_id=d7f07877b0f74fa296b0784f22c6fd79, region_id=RegionOne) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:46.545202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcff061-022b-4b6e-b6e0-c1e14bb91e5a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:46.546586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcff061-022b-4b6e-b6e0-c1e14bb91e5a None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:46.551505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 647/1300] 10.222.0.23 () {56 vars in 1251 bytes} [Tue Jul 12 13:17:46 2022] GET /identity/v3/endpoints?service_id=d7f07877b0f74fa296b0784f22c6fd79&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:46.557200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a3c81-d398-440b-96fe-4aa28755bb13 None admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:46.636547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:46.654494 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44a3c81-d398-440b-96fe-4aa28755bb13 None admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:46.687554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b066c6f2-19b1-4bac-844b-5082a7af5f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:46.688207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066c6f2-19b1-4bac-844b-5082a7af5f44 None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:46.688488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066c6f2-19b1-4bac-844b-5082a7af5f44 None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:46.689016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b066c6f2-19b1-4bac-844b-5082a7af5f44 None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:46.689730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066c6f2-19b1-4bac-844b-5082a7af5f44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:46.691083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b066c6f2-19b1-4bac-844b-5082a7af5f44 None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:46.697689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 654/1301] 10.222.0.23 () {56 vars in 1087 bytes} [Tue Jul 12 13:17:46 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:17:48.683073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a412a1-5ffc-4236-9fc7-f96cfe71720e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:48.683573 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a412a1-5ffc-4236-9fc7-f96cfe71720e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:48.683573 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a412a1-5ffc-4236-9fc7-f96cfe71720e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:48.685707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 648/1302] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:17:48.690978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30473b-c9a1-47af-b89c-44f1718bfdf5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:48.691412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30473b-c9a1-47af-b89c-44f1718bfdf5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:48.691519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f30473b-c9a1-47af-b89c-44f1718bfdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:48.707295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3f30473b-c9a1-47af-b89c-44f1718bfdf5 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:17:48.708483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3f30473b-c9a1-47af-b89c-44f1718bfdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:17:48.772740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 655/1303] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:17:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 12 13:17:48.796342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a412a1-5ffc-4236-9fc7-f96cfe71720e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:48.887885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:48.894893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a412a1-5ffc-4236-9fc7-f96cfe71720e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:48.931520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4b1c-9299-4353-a039-d3199e69b9b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:48.932319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4b1c-9299-4353-a039-d3199e69b9b6 None admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:48.932602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4b1c-9299-4353-a039-d3199e69b9b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:48.932918 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263c4b1c-9299-4353-a039-d3199e69b9b6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:48.934030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c4b1c-9299-4353-a039-d3199e69b9b6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:48.935654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263c4b1c-9299-4353-a039-d3199e69b9b6 None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application [None req-263c4b1c-9299-4353-a039-d3199e69b9b6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 12 13:17:48.949623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 12 13:17:48.950780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.get_or_create( Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application with Lock( Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self._enter() Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application return self.creator() Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application created_value = creator( Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: ERROR keystone.server.flask.application Jul 12 13:17:48.951680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 649/1304] 10.222.0.23 () {56 vars in 1117 bytes} [Tue Jul 12 13:17:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 12 13:17:48.957634 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30473b-c9a1-47af-b89c-44f1718bfdf5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:49.034041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:49.056962 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f30473b-c9a1-47af-b89c-44f1718bfdf5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:49.088661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333cfa1-3ed0-4deb-92e8-35d66d66ac7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:49.089321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333cfa1-3ed0-4deb-92e8-35d66d66ac7c None admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:49.089633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333cfa1-3ed0-4deb-92e8-35d66d66ac7c None admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:49.089913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333cfa1-3ed0-4deb-92e8-35d66d66ac7c None admin] PATH_INFO: `/v3/services` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:49.091376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333cfa1-3ed0-4deb-92e8-35d66d66ac7c None admin] RBAC: Adding query filter params (name=placement) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:17:49.091690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333cfa1-3ed0-4deb-92e8-35d66d66ac7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:49.093056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9333cfa1-3ed0-4deb-92e8-35d66d66ac7c None admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:49.098079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 656/1305] 10.222.0.23 () {56 vars in 1116 bytes} [Tue Jul 12 13:17:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:17:49.103925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4b1c-9299-4353-a039-d3199e69b9b6 None admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:17:49.201609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263c4b1c-9299-4353-a039-d3199e69b9b6 None admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:17:49.237348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691267d4-965b-424a-ae13-63715442dcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:17:49.238024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691267d4-965b-424a-ae13-63715442dcaf None admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:17:49.238309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691267d4-965b-424a-ae13-63715442dcaf None admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:17:49.238585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691267d4-965b-424a-ae13-63715442dcaf None admin] PATH_INFO: `/v3/endpoints` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:17:49.239324 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691267d4-965b-424a-ae13-63715442dcaf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:17:49.240679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691267d4-965b-424a-ae13-63715442dcaf None admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:17:49.258557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 650/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Tue Jul 12 13:17:49 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:17:59.095139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=30 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:17:59.256063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:18:51.820396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebb4e3-6f4d-4ff2-8d3a-76ad99d69cc4 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:51.820980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebb4e3-6f4d-4ff2-8d3a-76ad99d69cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:51.820980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebb4e3-6f4d-4ff2-8d3a-76ad99d69cc4 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:51.827084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 657/1307] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:18:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:18:51.828739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c42b-96f2-41dc-8514-1caee9e16c45 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:51.829644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c42b-96f2-41dc-8514-1caee9e16c45 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:51.830241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d8c42b-96f2-41dc-8514-1caee9e16c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:51.928134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b4d8c42b-96f2-41dc-8514-1caee9e16c45 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:18:51.929781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b4d8c42b-96f2-41dc-8514-1caee9e16c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:52.133981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 651/1308] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:18:52.188384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34cb71-e3b5-41eb-b8e1-82ed62acdecb None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:52.188733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34cb71-e3b5-41eb-b8e1-82ed62acdecb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:52.188939 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34cb71-e3b5-41eb-b8e1-82ed62acdecb None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:52.190877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 658/1309] 10.222.0.23 () {54 vars in 964 bytes} [Tue Jul 12 13:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:18:52.199416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d89277-d6bf-4402-b685-2c8ac0632ece None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:52.200219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d89277-d6bf-4402-b685-2c8ac0632ece None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:52.200219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d89277-d6bf-4402-b685-2c8ac0632ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:52.280833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-92d89277-d6bf-4402-b685-2c8ac0632ece None None] MFA Rules not processed for user `83837e4f8f9b4e538658876cfb56db16`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:18:52.282067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-92d89277-d6bf-4402-b685-2c8ac0632ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:52.303475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:18:52.364164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 652/1310] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Jul 12 13:18:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:18:52.373007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34cb71-e3b5-41eb-b8e1-82ed62acdecb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:18:52.374788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bc34cb71-e3b5-41eb-b8e1-82ed62acdecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:52.494915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34cb71-e3b5-41eb-b8e1-82ed62acdecb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:18:52.575734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111bfa04-dc97-4195-9056-4ef9970d110d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:18:52.576387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bfa04-dc97-4195-9056-4ef9970d110d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:52.576669 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bfa04-dc97-4195-9056-4ef9970d110d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:52.576943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111bfa04-dc97-4195-9056-4ef9970d110d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:52.578609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-111bfa04-dc97-4195-9056-4ef9970d110d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:52.634015 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:18:52.693674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bfa04-dc97-4195-9056-4ef9970d110d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:18:52.695238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111bfa04-dc97-4195-9056-4ef9970d110d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:18:52.716296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 659/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:18:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:18:55.069923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011ef81-7f1b-446a-bed4-88204327fd51 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:55.070503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011ef81-7f1b-446a-bed4-88204327fd51 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:55.070503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011ef81-7f1b-446a-bed4-88204327fd51 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:55.071880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 653/1312] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:18:55.077902 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11927d89-ea98-460c-b36d-6b0eef2971d3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:55.078329 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11927d89-ea98-460c-b36d-6b0eef2971d3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:55.078469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11927d89-ea98-460c-b36d-6b0eef2971d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:55.160454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-11927d89-ea98-460c-b36d-6b0eef2971d3 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:18:55.161737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-11927d89-ea98-460c-b36d-6b0eef2971d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:55.286425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 660/1313] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:18:55.325137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011ef81-7f1b-446a-bed4-88204327fd51 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:18:55.326635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a011ef81-7f1b-446a-bed4-88204327fd51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:55.449502 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011ef81-7f1b-446a-bed4-88204327fd51 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:18:55.548476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:18:55.551268 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4033f769-b94b-453c-bb51-577504967300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:18:55.551925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033f769-b94b-453c-bb51-577504967300 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:55.552214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033f769-b94b-453c-bb51-577504967300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:55.553542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4033f769-b94b-453c-bb51-577504967300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:55.554313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4033f769-b94b-453c-bb51-577504967300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:55.704080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033f769-b94b-453c-bb51-577504967300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:18:55.705579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4033f769-b94b-453c-bb51-577504967300 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:18:55.728348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 654/1314] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:18:58.123826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1446c-1182-4a90-99a9-31bc8b560def None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:58.124390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1446c-1182-4a90-99a9-31bc8b560def None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:58.124390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd1446c-1182-4a90-99a9-31bc8b560def None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:58.125828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 661/1315] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:18:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:18:58.130995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf578c1b-cf8a-441d-bf77-d8735b0cc11e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:58.131388 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf578c1b-cf8a-441d-bf77-d8735b0cc11e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:58.131568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf578c1b-cf8a-441d-bf77-d8735b0cc11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:58.195028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cf578c1b-cf8a-441d-bf77-d8735b0cc11e None None] MFA Rules not processed for user `4bd8fa54875c432abf00243484dacb64`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:18:58.196237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cf578c1b-cf8a-441d-bf77-d8735b0cc11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:58.247847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:18:58.296896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 655/1316] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:18:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:18:58.335899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1446c-1182-4a90-99a9-31bc8b560def None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:18:58.337459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bdd1446c-1182-4a90-99a9-31bc8b560def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:58.347553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:18:58.438854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd1446c-1182-4a90-99a9-31bc8b560def None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:18:58.509443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c697adbf-8ef5-4f0e-8d21-94fda1cdbd8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:18:58.510084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697adbf-8ef5-4f0e-8d21-94fda1cdbd8f service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:18:58.510365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697adbf-8ef5-4f0e-8d21-94fda1cdbd8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:18:58.510635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c697adbf-8ef5-4f0e-8d21-94fda1cdbd8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:18:58.513063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c697adbf-8ef5-4f0e-8d21-94fda1cdbd8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:18:58.574510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:18:58.619689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697adbf-8ef5-4f0e-8d21-94fda1cdbd8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:18:58.621826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c697adbf-8ef5-4f0e-8d21-94fda1cdbd8f service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:18:58.642132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 662/1317] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:18:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:01.177336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6bd10-5106-441c-9ed2-5faf70ea2400 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:01.177977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6bd10-5106-441c-9ed2-5faf70ea2400 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:01.177977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6bd10-5106-441c-9ed2-5faf70ea2400 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:01.179267 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 656/1318] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:01.185582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7262b-403e-49ab-bb2f-838f80904d7f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:01.186029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7262b-403e-49ab-bb2f-838f80904d7f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:01.186154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f7262b-403e-49ab-bb2f-838f80904d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:01.262493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-97f7262b-403e-49ab-bb2f-838f80904d7f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:01.263701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-97f7262b-403e-49ab-bb2f-838f80904d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:01.357949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 663/1319] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:01.408113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca9070-d2c3-4946-b847-f02c98166f0c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:01.408400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca9070-d2c3-4946-b847-f02c98166f0c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:01.408677 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca9070-d2c3-4946-b847-f02c98166f0c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:01.410082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 657/1320] 10.222.0.23 () {54 vars in 964 bytes} [Tue Jul 12 13:19:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:01.418541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc3b16-50cb-4db4-ba59-1c7976459e67 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:01.418829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc3b16-50cb-4db4-ba59-1c7976459e67 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:01.419109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbc3b16-50cb-4db4-ba59-1c7976459e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:01.439767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:01.492275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3dbc3b16-50cb-4db4-ba59-1c7976459e67 None None] MFA Rules not processed for user `83837e4f8f9b4e538658876cfb56db16`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:01.493498 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3dbc3b16-50cb-4db4-ba59-1c7976459e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:01.566566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 664/1321] 10.222.0.23 () {58 vars in 1061 bytes} [Tue Jul 12 13:19:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:01.575807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca9070-d2c3-4946-b847-f02c98166f0c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:01.577341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f2ca9070-d2c3-4946-b847-f02c98166f0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:01.679159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca9070-d2c3-4946-b847-f02c98166f0c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:01.751228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1800b4-c682-4e1d-aadd-d5d7937b4040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:01.751880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1800b4-c682-4e1d-aadd-d5d7937b4040 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:01.752169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1800b4-c682-4e1d-aadd-d5d7937b4040 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:01.752450 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1800b4-c682-4e1d-aadd-d5d7937b4040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:01.754145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3f1800b4-c682-4e1d-aadd-d5d7937b4040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:01.761845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:01.863155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1800b4-c682-4e1d-aadd-d5d7937b4040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:01.864588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1800b4-c682-4e1d-aadd-d5d7937b4040 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:01.882518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 658/1322] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:01 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:05.011511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784534e-a1e0-4c00-8ec3-f48dd5744df9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:05.012083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784534e-a1e0-4c00-8ec3-f48dd5744df9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:05.012083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9784534e-a1e0-4c00-8ec3-f48dd5744df9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:05.013601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 665/1323] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:05.018557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef956-199c-4776-8d65-1049111917f0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:05.018981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef956-199c-4776-8d65-1049111917f0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:05.019241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991ef956-199c-4776-8d65-1049111917f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:05.087420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-991ef956-199c-4776-8d65-1049111917f0 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:05.088629 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-991ef956-199c-4776-8d65-1049111917f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:05.146273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:05.191249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 659/1324] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:05.227926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784534e-a1e0-4c00-8ec3-f48dd5744df9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:05.229518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9784534e-a1e0-4c00-8ec3-f48dd5744df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:05.340030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9784534e-a1e0-4c00-8ec3-f48dd5744df9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:05.352370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:05.413155 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6add91cf-9066-42ce-bedf-cfd19fc79476 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:05.413867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add91cf-9066-42ce-bedf-cfd19fc79476 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:05.414155 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add91cf-9066-42ce-bedf-cfd19fc79476 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:05.414432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6add91cf-9066-42ce-bedf-cfd19fc79476 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:05.416102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6add91cf-9066-42ce-bedf-cfd19fc79476 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:05.525275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add91cf-9066-42ce-bedf-cfd19fc79476 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:05.526785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6add91cf-9066-42ce-bedf-cfd19fc79476 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:05.545052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 666/1325] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:07.650823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08042d-5a00-4d4b-b6a3-9e33c47a5edc None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:07.651346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08042d-5a00-4d4b-b6a3-9e33c47a5edc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:07.651346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08042d-5a00-4d4b-b6a3-9e33c47a5edc None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:07.653347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 660/1326] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:07.658565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf87947-5e9f-429f-a07b-e16c04e7486a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:07.658981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf87947-5e9f-429f-a07b-e16c04e7486a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:07.659122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf87947-5e9f-429f-a07b-e16c04e7486a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:07.705751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:07.724594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5cf87947-5e9f-429f-a07b-e16c04e7486a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:07.725843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5cf87947-5e9f-429f-a07b-e16c04e7486a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:07.822831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 667/1327] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:07.859770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08042d-5a00-4d4b-b6a3-9e33c47a5edc None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:07.861301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8c08042d-5a00-4d4b-b6a3-9e33c47a5edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:07.970390 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08042d-5a00-4d4b-b6a3-9e33c47a5edc None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:08.044123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef1b0f3-239e-4d10-b403-5ac75e94e0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:08.044783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1b0f3-239e-4d10-b403-5ac75e94e0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:08.045072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1b0f3-239e-4d10-b403-5ac75e94e0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:08.045352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef1b0f3-239e-4d10-b403-5ac75e94e0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:08.047135 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1ef1b0f3-239e-4d10-b403-5ac75e94e0a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:08.068049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:08.164110 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1b0f3-239e-4d10-b403-5ac75e94e0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:08.165538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef1b0f3-239e-4d10-b403-5ac75e94e0a6 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:08.183463 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 661/1328] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:10.292244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89502906-1a70-4e89-96a4-774d16cf044d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:10.292802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89502906-1a70-4e89-96a4-774d16cf044d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:10.292967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89502906-1a70-4e89-96a4-774d16cf044d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:10.294524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 668/1329] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:10.299636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa920569-311d-41f8-af7a-09089dbc3509 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:10.300043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa920569-311d-41f8-af7a-09089dbc3509 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:10.300210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa920569-311d-41f8-af7a-09089dbc3509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:10.370510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fa920569-311d-41f8-af7a-09089dbc3509 None None] MFA Rules not processed for user `4bd8fa54875c432abf00243484dacb64`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:10.371668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fa920569-311d-41f8-af7a-09089dbc3509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:10.438031 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:10.473804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 662/1330] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:10 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:10.511641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89502906-1a70-4e89-96a4-774d16cf044d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:10.513178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-89502906-1a70-4e89-96a4-774d16cf044d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:10.615495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89502906-1a70-4e89-96a4-774d16cf044d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:10.636025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:10.693328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41a97f-671b-45e8-8ec7-2edc7446632a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:10.694013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41a97f-671b-45e8-8ec7-2edc7446632a service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:10.694302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41a97f-671b-45e8-8ec7-2edc7446632a service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:10.694585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41a97f-671b-45e8-8ec7-2edc7446632a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:10.696256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1d41a97f-671b-45e8-8ec7-2edc7446632a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:10.807607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41a97f-671b-45e8-8ec7-2edc7446632a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:10.809741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41a97f-671b-45e8-8ec7-2edc7446632a service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:10.836973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 669/1331] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:10 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:13.142198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc557aa-66e2-4e1b-8ecf-1698881bed6c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:13.142765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc557aa-66e2-4e1b-8ecf-1698881bed6c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:13.142765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc557aa-66e2-4e1b-8ecf-1698881bed6c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:13.144117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 663/1332] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:13.149269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a288-791f-4110-8a21-446a26bb2618 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:13.149739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a288-791f-4110-8a21-446a26bb2618 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:13.149834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a288-791f-4110-8a21-446a26bb2618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:13.210740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:13.213555 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7dc4a288-791f-4110-8a21-446a26bb2618 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:13.215514 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7dc4a288-791f-4110-8a21-446a26bb2618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:13.312338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 670/1333] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:13.348141 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc557aa-66e2-4e1b-8ecf-1698881bed6c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:13.349743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5cc557aa-66e2-4e1b-8ecf-1698881bed6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:13.453280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc557aa-66e2-4e1b-8ecf-1698881bed6c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:13.524324 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71424187-9ad3-4c57-a0ec-bd6ebb091805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:13.524958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71424187-9ad3-4c57-a0ec-bd6ebb091805 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:13.525246 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71424187-9ad3-4c57-a0ec-bd6ebb091805 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:13.525614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71424187-9ad3-4c57-a0ec-bd6ebb091805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:13.527843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-71424187-9ad3-4c57-a0ec-bd6ebb091805 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:13.554007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:13.638872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71424187-9ad3-4c57-a0ec-bd6ebb091805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:13.640314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71424187-9ad3-4c57-a0ec-bd6ebb091805 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:13.659174 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 664/1334] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:15.765632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582aae-af17-4991-ad96-c9d6599f1d6c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:15.766260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582aae-af17-4991-ad96-c9d6599f1d6c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:15.766260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582aae-af17-4991-ad96-c9d6599f1d6c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:15.767538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 671/1335] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:15.772656 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eca46-5051-41f5-8f5b-1f16a2ecfd25 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:15.773044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eca46-5051-41f5-8f5b-1f16a2ecfd25 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:15.773215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399eca46-5051-41f5-8f5b-1f16a2ecfd25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:15.836030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-399eca46-5051-41f5-8f5b-1f16a2ecfd25 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:15.837238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-399eca46-5051-41f5-8f5b-1f16a2ecfd25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:15.911009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:15.933985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 665/1336] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:15.974002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582aae-af17-4991-ad96-c9d6599f1d6c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:15.975530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0b582aae-af17-4991-ad96-c9d6599f1d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:16.078443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582aae-af17-4991-ad96-c9d6599f1d6c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:16.107193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:16.151059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19920698-38c7-4e95-89ca-545460167a9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:16.152357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19920698-38c7-4e95-89ca-545460167a9e service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:16.152642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19920698-38c7-4e95-89ca-545460167a9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:16.152916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19920698-38c7-4e95-89ca-545460167a9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:16.154590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-19920698-38c7-4e95-89ca-545460167a9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:16.261565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19920698-38c7-4e95-89ca-545460167a9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:16.262997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19920698-38c7-4e95-89ca-545460167a9e service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:16.280945 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 672/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:18.691295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9767a-4246-41e3-80a5-6fc79a8ab828 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:18.691829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9767a-4246-41e3-80a5-6fc79a8ab828 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:18.691829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f9767a-4246-41e3-80a5-6fc79a8ab828 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:18.693124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 666/1338] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:18.698915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf8ef-39cb-49e0-b859-499bd4f81f3b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:18.699480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf8ef-39cb-49e0-b859-499bd4f81f3b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:18.699605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71adf8ef-39cb-49e0-b859-499bd4f81f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:18.767771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-71adf8ef-39cb-49e0-b859-499bd4f81f3b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:18.769030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-71adf8ef-39cb-49e0-b859-499bd4f81f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:18.777294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:18.863374 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 673/1339] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:18.899180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9767a-4246-41e3-80a5-6fc79a8ab828 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:18.900769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-45f9767a-4246-41e3-80a5-6fc79a8ab828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:19.009304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f9767a-4246-41e3-80a5-6fc79a8ab828 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:19.081138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d369da-b73c-4cec-bfb2-439602b7705a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:19.081811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d369da-b73c-4cec-bfb2-439602b7705a service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:19.082095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d369da-b73c-4cec-bfb2-439602b7705a service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:19.082369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d369da-b73c-4cec-bfb2-439602b7705a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:19.083994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f4d369da-b73c-4cec-bfb2-439602b7705a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:19.118718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:19.189819 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d369da-b73c-4cec-bfb2-439602b7705a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:19.191539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d369da-b73c-4cec-bfb2-439602b7705a service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:19.209957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 667/1340] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:19:23.172752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6524f-a38e-4f65-a0c9-c5658b6ccb57 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:23.173306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6524f-a38e-4f65-a0c9-c5658b6ccb57 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:23.173306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c6524f-a38e-4f65-a0c9-c5658b6ccb57 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:23.175060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 674/1341] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:23.180106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0199382-b734-47a8-8442-8906cfe5655d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:23.180531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0199382-b734-47a8-8442-8906cfe5655d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:23.180668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0199382-b734-47a8-8442-8906cfe5655d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:23.249589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f0199382-b734-47a8-8442-8906cfe5655d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:23.250818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f0199382-b734-47a8-8442-8906cfe5655d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:23.337594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:23.349540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 668/1342] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:23.394292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6524f-a38e-4f65-a0c9-c5658b6ccb57 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:23.395829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f0c6524f-a38e-4f65-a0c9-c5658b6ccb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:23.503833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c6524f-a38e-4f65-a0c9-c5658b6ccb57 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:23.543958 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:23.583397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb02eb5-0f77-4f6e-bb0a-4c23f39683e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:23.584048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb02eb5-0f77-4f6e-bb0a-4c23f39683e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:23.584333 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb02eb5-0f77-4f6e-bb0a-4c23f39683e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:23.584609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb02eb5-0f77-4f6e-bb0a-4c23f39683e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:23.586288 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2bb02eb5-0f77-4f6e-bb0a-4c23f39683e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:23.696665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb02eb5-0f77-4f6e-bb0a-4c23f39683e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:23.698136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb02eb5-0f77-4f6e-bb0a-4c23f39683e9 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:23.719506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 675/1343] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:26.335800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd99fe2-c080-42b8-8ab2-e39f51fce15e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:26.336411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd99fe2-c080-42b8-8ab2-e39f51fce15e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:26.336411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd99fe2-c080-42b8-8ab2-e39f51fce15e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:26.337760 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 669/1344] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:26.343794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ba4ad-37f5-4e38-8ea5-96be77771e9b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:26.344385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ba4ad-37f5-4e38-8ea5-96be77771e9b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:26.344385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ba4ad-37f5-4e38-8ea5-96be77771e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:26.413305 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8c7ba4ad-37f5-4e38-8ea5-96be77771e9b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:26.414583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8c7ba4ad-37f5-4e38-8ea5-96be77771e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:26.434090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:26.517708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 676/1345] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:26 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:26.556741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd99fe2-c080-42b8-8ab2-e39f51fce15e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:26.558372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dcd99fe2-c080-42b8-8ab2-e39f51fce15e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:26.660137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd99fe2-c080-42b8-8ab2-e39f51fce15e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:26.732419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a7e73b-5924-4501-b8a8-2c6befeaf7c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:26.733065 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e73b-5924-4501-b8a8-2c6befeaf7c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:26.733347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e73b-5924-4501-b8a8-2c6befeaf7c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:26.733648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a7e73b-5924-4501-b8a8-2c6befeaf7c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:26.735289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-24a7e73b-5924-4501-b8a8-2c6befeaf7c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:26.778343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:26.846076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7e73b-5924-4501-b8a8-2c6befeaf7c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:26.847516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a7e73b-5924-4501-b8a8-2c6befeaf7c0 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:26.865118 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 670/1346] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:26 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:30.964845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32984f-a402-4c4e-8ce7-68530f443672 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:30.965477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32984f-a402-4c4e-8ce7-68530f443672 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:30.965477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32984f-a402-4c4e-8ce7-68530f443672 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:30.966948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 677/1347] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:30.972072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba71d7-af6d-44fd-a27b-07ca46c55d40 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:30.972521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba71d7-af6d-44fd-a27b-07ca46c55d40 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:30.972623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ba71d7-af6d-44fd-a27b-07ca46c55d40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:31.040093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f6ba71d7-af6d-44fd-a27b-07ca46c55d40 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:31.041338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f6ba71d7-af6d-44fd-a27b-07ca46c55d40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:31.165301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:31.169599 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 671/1348] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:31.206489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32984f-a402-4c4e-8ce7-68530f443672 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:31.208041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0a32984f-a402-4c4e-8ce7-68530f443672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:31.318578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32984f-a402-4c4e-8ce7-68530f443672 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:31.369061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:31.396695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca7994f-ae38-445c-872d-3ad6dfa8b3b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:31.397362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7994f-ae38-445c-872d-3ad6dfa8b3b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:31.397727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7994f-ae38-445c-872d-3ad6dfa8b3b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:31.398006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca7994f-ae38-445c-872d-3ad6dfa8b3b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:31.399680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2ca7994f-ae38-445c-872d-3ad6dfa8b3b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:31.514971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7994f-ae38-445c-872d-3ad6dfa8b3b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:31.516426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca7994f-ae38-445c-872d-3ad6dfa8b3b5 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:31.536033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 678/1349] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:36.529314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f54a8-cdda-43f3-b515-472d8d4f3114 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:36.537688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f54a8-cdda-43f3-b515-472d8d4f3114 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:36.537983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356f54a8-cdda-43f3-b515-472d8d4f3114 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:36.539909 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 672/1350] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:19:36 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:36.545114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496b222-f926-496a-983d-a9397148d100 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:36.545630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496b222-f926-496a-983d-a9397148d100 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:36.545731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6496b222-f926-496a-983d-a9397148d100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:36.633983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6496b222-f926-496a-983d-a9397148d100 None None] MFA Rules not processed for user `2e97119806774c2482a9886b11ab6f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:36.635219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6496b222-f926-496a-983d-a9397148d100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:36.656237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:36.772860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 679/1351] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:19:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:40.888480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314355-4d65-4bb6-a384-78716aaa0087 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:40.889403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314355-4d65-4bb6-a384-78716aaa0087 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:40.889699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73314355-4d65-4bb6-a384-78716aaa0087 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:40.891223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 673/1352] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:19:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:40.897489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05054b10-daa0-4dec-8c0d-9785350a6648 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:40.897926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05054b10-daa0-4dec-8c0d-9785350a6648 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:40.898181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05054b10-daa0-4dec-8c0d-9785350a6648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:40.978966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-05054b10-daa0-4dec-8c0d-9785350a6648 None None] MFA Rules not processed for user `2e97119806774c2482a9886b11ab6f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:40.980511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-05054b10-daa0-4dec-8c0d-9785350a6648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:41.055219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:41.094003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 680/1353] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:19:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:19:42.440534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f58040-c9d8-4881-b256-13fe9f29f04a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:42.441313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f58040-c9d8-4881-b256-13fe9f29f04a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:42.441313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f58040-c9d8-4881-b256-13fe9f29f04a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:42.442708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 674/1354] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:19:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:42.447588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9df8d-aba2-4b0f-b6e6-9dfa7fb06742 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:42.447999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9df8d-aba2-4b0f-b6e6-9dfa7fb06742 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:42.448288 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c9df8d-aba2-4b0f-b6e6-9dfa7fb06742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:42.522639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-77c9df8d-aba2-4b0f-b6e6-9dfa7fb06742 None None] MFA Rules not processed for user `2e97119806774c2482a9886b11ab6f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:42.523746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-77c9df8d-aba2-4b0f-b6e6-9dfa7fb06742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:42.631030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 681/1355] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:19:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:46.165310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c378a4-be77-4c41-9ea8-1675c01604ac None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:46.166213 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c378a4-be77-4c41-9ea8-1675c01604ac None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:46.166659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c378a4-be77-4c41-9ea8-1675c01604ac None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:46.175570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9dd08-088e-429c-bbdd-4fa46c10c902 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:46.176509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9dd08-088e-429c-bbdd-4fa46c10c902 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:46.176960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9dd08-088e-429c-bbdd-4fa46c10c902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:46.180694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 675/1356] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:46 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:46.216694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:46.259698 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6cc9dd08-088e-429c-bbdd-4fa46c10c902 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:46.261767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6cc9dd08-088e-429c-bbdd-4fa46c10c902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:46.363229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 682/1357] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:46.418281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc270ee3-20a4-411f-a95a-391503b0e6fe None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:46.418867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc270ee3-20a4-411f-a95a-391503b0e6fe None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:46.419412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc270ee3-20a4-411f-a95a-391503b0e6fe None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:46.422695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 676/1358] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jul 12 13:19:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:46.434789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ad909-c407-4eee-bae8-af5c11ba4360 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:46.435016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ad909-c407-4eee-bae8-af5c11ba4360 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:46.435369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37ad909-c407-4eee-bae8-af5c11ba4360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:46.528317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e37ad909-c407-4eee-bae8-af5c11ba4360 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:46.529803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e37ad909-c407-4eee-bae8-af5c11ba4360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:46.584166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:46.660083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 683/1359] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jul 12 13:19:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:46.667965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc270ee3-20a4-411f-a95a-391503b0e6fe None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:46.669671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cc270ee3-20a4-411f-a95a-391503b0e6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:46.820270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc270ee3-20a4-411f-a95a-391503b0e6fe None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:46.896668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:46.940131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c66eb6-423d-4011-8c95-240ef71ad44c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:46.940786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66eb6-423d-4011-8c95-240ef71ad44c service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:46.941329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66eb6-423d-4011-8c95-240ef71ad44c service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:46.941636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c66eb6-423d-4011-8c95-240ef71ad44c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:46.943318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-31c66eb6-423d-4011-8c95-240ef71ad44c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:47.048344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c66eb6-423d-4011-8c95-240ef71ad44c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:47.049916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c66eb6-423d-4011-8c95-240ef71ad44c service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:47.069163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 677/1360] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:19:49.339832 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960566fd-db50-4ea7-866b-7fac16dc2169 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:49.340391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960566fd-db50-4ea7-866b-7fac16dc2169 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:49.340391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960566fd-db50-4ea7-866b-7fac16dc2169 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:49.341923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 684/1361] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:49.346979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26db589-09e6-48d2-ad1d-e6134f7b1eb5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:49.347411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26db589-09e6-48d2-ad1d-e6134f7b1eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:49.347579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26db589-09e6-48d2-ad1d-e6134f7b1eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:49.415027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b26db589-09e6-48d2-ad1d-e6134f7b1eb5 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:49.416217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b26db589-09e6-48d2-ad1d-e6134f7b1eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:49.457561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:49.525563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 678/1362] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:19:49.571456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960566fd-db50-4ea7-866b-7fac16dc2169 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:49.573161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-960566fd-db50-4ea7-866b-7fac16dc2169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:49.747028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960566fd-db50-4ea7-866b-7fac16dc2169 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:49.774349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:49.898495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba6933-8bcf-4cbc-ae40-948630abe3b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:49.899154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba6933-8bcf-4cbc-ae40-948630abe3b8 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:49.899441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba6933-8bcf-4cbc-ae40-948630abe3b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:49.899721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba6933-8bcf-4cbc-ae40-948630abe3b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:49.902109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-26ba6933-8bcf-4cbc-ae40-948630abe3b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:50.002984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba6933-8bcf-4cbc-ae40-948630abe3b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:50.004596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba6933-8bcf-4cbc-ae40-948630abe3b8 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:50.025508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 685/1363] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:52.423178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae98c7-c373-4b62-b496-eaba9602aa1a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:52.423694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae98c7-c373-4b62-b496-eaba9602aa1a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:52.423830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ae98c7-c373-4b62-b496-eaba9602aa1a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:52.425286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 679/1364] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:52.430653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517586e-9559-4cf6-b2a7-bc7e7b818fb7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:52.431098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517586e-9559-4cf6-b2a7-bc7e7b818fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:52.431202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4517586e-9559-4cf6-b2a7-bc7e7b818fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:52.453710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:52.514033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4517586e-9559-4cf6-b2a7-bc7e7b818fb7 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:52.515417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4517586e-9559-4cf6-b2a7-bc7e7b818fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:52.617469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 686/1365] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:52.662383 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829567a5-21e9-492e-bfb7-9af182e6750d None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:52.662678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829567a5-21e9-492e-bfb7-9af182e6750d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:52.663148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829567a5-21e9-492e-bfb7-9af182e6750d None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:52.664503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 680/1366] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jul 12 13:19:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:52.671306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b229570-36fd-45f3-bd17-f60eeff6c1b7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:52.671668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b229570-36fd-45f3-bd17-f60eeff6c1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:52.672054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b229570-36fd-45f3-bd17-f60eeff6c1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:52.748999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7b229570-36fd-45f3-bd17-f60eeff6c1b7 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:52.750265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7b229570-36fd-45f3-bd17-f60eeff6c1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:52.787994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:52.880464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 687/1367] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jul 12 13:19:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:52.889466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829567a5-21e9-492e-bfb7-9af182e6750d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:52.891161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-829567a5-21e9-492e-bfb7-9af182e6750d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:53.060696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829567a5-21e9-492e-bfb7-9af182e6750d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:53.083235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:53.186371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a66686-b77c-46a4-84db-5724e259cb78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:53.187028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66686-b77c-46a4-84db-5724e259cb78 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:53.187314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66686-b77c-46a4-84db-5724e259cb78 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:53.187593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a66686-b77c-46a4-84db-5724e259cb78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:53.189405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f7a66686-b77c-46a4-84db-5724e259cb78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:53.295587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a66686-b77c-46a4-84db-5724e259cb78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:53.297218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a66686-b77c-46a4-84db-5724e259cb78 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:53.317055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 681/1368] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:54.386317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfe26f-285e-40d0-bb2d-4bd52256fc75 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:54.386839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfe26f-285e-40d0-bb2d-4bd52256fc75 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:54.386919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cfe26f-285e-40d0-bb2d-4bd52256fc75 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:54.388429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 688/1369] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:54.393477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2d865-9c8d-4139-9fec-ae0fe82ae31e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:54.393889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2d865-9c8d-4139-9fec-ae0fe82ae31e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:54.394079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b2d865-9c8d-4139-9fec-ae0fe82ae31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:54.426195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:54.463321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-20b2d865-9c8d-4139-9fec-ae0fe82ae31e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:54.464578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-20b2d865-9c8d-4139-9fec-ae0fe82ae31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:54.576712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 682/1370] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:54.915021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65b262-fc46-4e60-b9fc-d98144262485 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:54.915021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65b262-fc46-4e60-b9fc-d98144262485 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:54.915021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65b262-fc46-4e60-b9fc-d98144262485 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:54.916339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 689/1371] 10.222.0.23 () {54 vars in 963 bytes} [Tue Jul 12 13:19:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:54.922128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d814-e5dd-4a23-ade5-07979d77ebfb None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:54.922539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d814-e5dd-4a23-ade5-07979d77ebfb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:54.922991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686d814-e5dd-4a23-ade5-07979d77ebfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:55.005354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5686d814-e5dd-4a23-ade5-07979d77ebfb None None] MFA Rules not processed for user `b4f12d08d7084387a1bea7865bf77d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:55.006746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5686d814-e5dd-4a23-ade5-07979d77ebfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:55.058749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:55.093702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 683/1372] 10.222.0.23 () {58 vars in 1060 bytes} [Tue Jul 12 13:19:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:55.101324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65b262-fc46-4e60-b9fc-d98144262485 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:55.103163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6d65b262-fc46-4e60-b9fc-d98144262485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:55.212405 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65b262-fc46-4e60-b9fc-d98144262485 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:55.260953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:55.297558 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:55.298208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:55.298490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:55.298802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:55.301799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:55.431949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:55.433796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:55.456363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d1fbd-3740-4907-b585-be31d5c13f68 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:55.456989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d1fbd-3740-4907-b585-be31d5c13f68 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:55.456989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d1fbd-3740-4907-b585-be31d5c13f68 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:55.462872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 684/1373] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:55.463531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bd433-69a2-40ab-9f90-9f624878b499 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:55.464432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bd433-69a2-40ab-9f90-9f624878b499 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:55.464933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bd433-69a2-40ab-9f90-9f624878b499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:55.465764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 690/1374] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:19:55.533776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-687bd433-69a2-40ab-9f90-9f624878b499 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:55.535009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-687bd433-69a2-40ab-9f90-9f624878b499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:55.634709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 685/1375] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:55.679354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:55.680580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:55.754040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:55.786204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6cf2a-6dac-4b22-8903-57924bb01b69 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:55.901748 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac641e2-ed0e-421f-981c-4c63822750a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:55.902500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac641e2-ed0e-421f-981c-4c63822750a6 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:55.902789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac641e2-ed0e-421f-981c-4c63822750a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:55.903061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac641e2-ed0e-421f-981c-4c63822750a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:55.904700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4ac641e2-ed0e-421f-981c-4c63822750a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:55.987384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:55.999193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac641e2-ed0e-421f-981c-4c63822750a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:56.000911 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac641e2-ed0e-421f-981c-4c63822750a6 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:56.019006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 691/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:19:58.188555 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86023d2-1827-402f-972a-1e979ed43d17 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:58.189052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86023d2-1827-402f-972a-1e979ed43d17 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:58.189052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86023d2-1827-402f-972a-1e979ed43d17 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:58.190427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 686/1377] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:19:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:19:58.196035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd75160-e868-435b-b27d-463577484b3b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:58.196481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd75160-e868-435b-b27d-463577484b3b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:58.196603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd75160-e868-435b-b27d-463577484b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:58.275605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ffd75160-e868-435b-b27d-463577484b3b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:19:58.277517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ffd75160-e868-435b-b27d-463577484b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:58.375499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 692/1378] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:19:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:19:58.423930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86023d2-1827-402f-972a-1e979ed43d17 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:19:58.429213 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b86023d2-1827-402f-972a-1e979ed43d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:58.483858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:58.599214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86023d2-1827-402f-972a-1e979ed43d17 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:19:58.712762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d7f27-249a-4fc7-b123-bf6ca1ac06d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:19:58.713477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d7f27-249a-4fc7-b123-bf6ca1ac06d1 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:19:58.714278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d7f27-249a-4fc7-b123-bf6ca1ac06d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:19:58.714679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d7f27-249a-4fc7-b123-bf6ca1ac06d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:19:58.716327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-205d7f27-249a-4fc7-b123-bf6ca1ac06d1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:19:58.727880 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:19:58.815243 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d7f27-249a-4fc7-b123-bf6ca1ac06d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:19:58.817133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d7f27-249a-4fc7-b123-bf6ca1ac06d1 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:19:58.835710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 687/1379] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:19:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:00.969719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861538d1-714c-425b-b019-7bbf8fc7bfab None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:00.970283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861538d1-714c-425b-b019-7bbf8fc7bfab None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:00.970283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861538d1-714c-425b-b019-7bbf8fc7bfab None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:00.971706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 693/1380] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:00.976839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc26b-8917-44ab-b5c4-5216139f8e64 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:00.977245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc26b-8917-44ab-b5c4-5216139f8e64 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:00.977450 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cc26b-8917-44ab-b5c4-5216139f8e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:01.048346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b94cc26b-8917-44ab-b5c4-5216139f8e64 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:01.049603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b94cc26b-8917-44ab-b5c4-5216139f8e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:01.152361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:01.167293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 688/1381] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:01.212647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861538d1-714c-425b-b019-7bbf8fc7bfab None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:01.215083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-861538d1-714c-425b-b019-7bbf8fc7bfab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:01.293458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:01.396363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861538d1-714c-425b-b019-7bbf8fc7bfab None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:01.522869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eb59fb-30fb-4704-b2ad-e7b1fb2b3a2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:01.523539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb59fb-30fb-4704-b2ad-e7b1fb2b3a2d service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:01.523892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb59fb-30fb-4704-b2ad-e7b1fb2b3a2d service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:01.524241 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eb59fb-30fb-4704-b2ad-e7b1fb2b3a2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:01.526063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-27eb59fb-30fb-4704-b2ad-e7b1fb2b3a2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:01.549140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:01.625275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb59fb-30fb-4704-b2ad-e7b1fb2b3a2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:01.627099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eb59fb-30fb-4704-b2ad-e7b1fb2b3a2d service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:01.648085 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 694/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:03.898482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d23a0-29fc-4750-b188-ca486c405c02 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:03.899022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d23a0-29fc-4750-b188-ca486c405c02 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:03.899022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d23a0-29fc-4750-b188-ca486c405c02 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:03.900332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 689/1383] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:20:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:03.906802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa603156-aaeb-423d-a056-acdc45569e74 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:03.907231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa603156-aaeb-423d-a056-acdc45569e74 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:03.907443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa603156-aaeb-423d-a056-acdc45569e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:03.985054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fa603156-aaeb-423d-a056-acdc45569e74 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:03.986326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fa603156-aaeb-423d-a056-acdc45569e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:04.091922 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:04.097525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 695/1384] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:20:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:04.146829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d23a0-29fc-4750-b188-ca486c405c02 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:04.148743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f72d23a0-29fc-4750-b188-ca486c405c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:04.319716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d23a0-29fc-4750-b188-ca486c405c02 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:04.387623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:04.445032 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66461c05-fa70-48d9-8f41-1ec78aa3fc03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:04.445616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66461c05-fa70-48d9-8f41-1ec78aa3fc03 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:04.445896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66461c05-fa70-48d9-8f41-1ec78aa3fc03 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:04.446177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66461c05-fa70-48d9-8f41-1ec78aa3fc03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:04.448486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-66461c05-fa70-48d9-8f41-1ec78aa3fc03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:04.572554 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66461c05-fa70-48d9-8f41-1ec78aa3fc03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:04.574335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66461c05-fa70-48d9-8f41-1ec78aa3fc03 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:04.592689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 690/1385] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:06.748579 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afeac-b454-4d89-9d3a-8787c2febb85 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:06.749165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afeac-b454-4d89-9d3a-8787c2febb85 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:06.749289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48afeac-b454-4d89-9d3a-8787c2febb85 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:06.751213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 696/1386] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:20:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:06.757363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c784c52-c7d2-4a89-88b2-4d4113243a93 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:06.757830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c784c52-c7d2-4a89-88b2-4d4113243a93 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:06.758040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c784c52-c7d2-4a89-88b2-4d4113243a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:06.836746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1c784c52-c7d2-4a89-88b2-4d4113243a93 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:06.838091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1c784c52-c7d2-4a89-88b2-4d4113243a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:06.865326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:06.943667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 691/1387] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:20:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:06.990986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48afeac-b454-4d89-9d3a-8787c2febb85 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:06.992537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b48afeac-b454-4d89-9d3a-8787c2febb85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:07.144351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48afeac-b454-4d89-9d3a-8787c2febb85 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:07.218206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:07.266822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f38d6d-d2b4-4d8d-9750-82a5706ee656 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:07.267478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f38d6d-d2b4-4d8d-9750-82a5706ee656 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:07.267763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f38d6d-d2b4-4d8d-9750-82a5706ee656 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:07.268038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f38d6d-d2b4-4d8d-9750-82a5706ee656 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:07.269731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-96f38d6d-d2b4-4d8d-9750-82a5706ee656 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:07.378297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f38d6d-d2b4-4d8d-9750-82a5706ee656 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:07.379920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f38d6d-d2b4-4d8d-9750-82a5706ee656 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:07.398806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 697/1388] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:09.505884 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac484fb-d7ec-46f0-bb47-fb2bc2b6a0a0 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:09.506409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac484fb-d7ec-46f0-bb47-fb2bc2b6a0a0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:09.506409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac484fb-d7ec-46f0-bb47-fb2bc2b6a0a0 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:09.507740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 692/1389] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:20:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:09.513658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189224c-7867-4ec7-bd5a-eedbfd0b706d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:09.514137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189224c-7867-4ec7-bd5a-eedbfd0b706d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:09.514245 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189224c-7867-4ec7-bd5a-eedbfd0b706d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:09.578962 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d189224c-7867-4ec7-bd5a-eedbfd0b706d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:09.580153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d189224c-7867-4ec7-bd5a-eedbfd0b706d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:09.614969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:09.678560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 698/1390] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:20:09.725206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac484fb-d7ec-46f0-bb47-fb2bc2b6a0a0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:09.727093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1ac484fb-d7ec-46f0-bb47-fb2bc2b6a0a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:09.874716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac484fb-d7ec-46f0-bb47-fb2bc2b6a0a0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:09.883446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:09.987716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359768be-cdef-4a31-b958-720537b176b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:09.988364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359768be-cdef-4a31-b958-720537b176b8 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:09.988649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359768be-cdef-4a31-b958-720537b176b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:09.988925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359768be-cdef-4a31-b958-720537b176b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:09.990621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-359768be-cdef-4a31-b958-720537b176b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:10.081871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359768be-cdef-4a31-b958-720537b176b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:10.083499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359768be-cdef-4a31-b958-720537b176b8 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:10.101733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 693/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:10.681891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea0d82-2f63-4778-b0be-a9a69833b64f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:10.682498 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea0d82-2f63-4778-b0be-a9a69833b64f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:10.683060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea0d82-2f63-4778-b0be-a9a69833b64f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:10.684455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 699/1392] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:10.690112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d6a08-4c21-4b07-8859-0a6e3c6efde3 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:10.690530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d6a08-4c21-4b07-8859-0a6e3c6efde3 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:10.690750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d6a08-4c21-4b07-8859-0a6e3c6efde3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:10.706200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:10.759297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-826d6a08-4c21-4b07-8859-0a6e3c6efde3 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:10.761684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-826d6a08-4c21-4b07-8859-0a6e3c6efde3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:10.861249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 694/1393] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:10.918480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ed88-deaa-4834-91d5-8a2b670a5bb0 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:10.918715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ed88-deaa-4834-91d5-8a2b670a5bb0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:10.919225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ed88-deaa-4834-91d5-8a2b670a5bb0 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:10.920743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 700/1394] 10.222.0.23 () {54 vars in 962 bytes} [Tue Jul 12 13:20:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:10.925913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579dcc3-f60d-4842-b08e-399187d21fa1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:10.926212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579dcc3-f60d-4842-b08e-399187d21fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:10.926494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579dcc3-f60d-4842-b08e-399187d21fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:11.002356 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-2579dcc3-f60d-4842-b08e-399187d21fa1 None None] MFA Rules not processed for user `0ff10db6937b4f2b889c30dd5fad8111`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:11.003569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2579dcc3-f60d-4842-b08e-399187d21fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:11.037340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:11.080308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 695/1395] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Jul 12 13:20:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:11.087812 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1ed88-deaa-4834-91d5-8a2b670a5bb0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:11.090146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b1d1ed88-deaa-4834-91d5-8a2b670a5bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:11.194338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1ed88-deaa-4834-91d5-8a2b670a5bb0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:11.265364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:11.277515 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb034c-7f1f-4983-ba10-f96fdf6c3d8f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ff10db6937b4f2b889c30dd5fad8111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:11.278172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb034c-7f1f-4983-ba10-f96fdf6c3d8f service glance] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:11.278457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb034c-7f1f-4983-ba10-f96fdf6c3d8f service glance] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:11.278734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb034c-7f1f-4983-ba10-f96fdf6c3d8f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:11.280377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f0eb034c-7f1f-4983-ba10-f96fdf6c3d8f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:11.390511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb034c-7f1f-4983-ba10-f96fdf6c3d8f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:11.391955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb034c-7f1f-4983-ba10-f96fdf6c3d8f service glance] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:11.410840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 701/1396] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:12.118969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48a43d-e5f1-432b-be6c-3c6a566e7e4d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:12.119440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48a43d-e5f1-432b-be6c-3c6a566e7e4d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:12.119763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad48a43d-e5f1-432b-be6c-3c6a566e7e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:12.186262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ad48a43d-e5f1-432b-be6c-3c6a566e7e4d None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:12.187485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ad48a43d-e5f1-432b-be6c-3c6a566e7e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:12.199306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2f796-dd0b-44fe-8d8d-7892a71edd3f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:12.199800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2f796-dd0b-44fe-8d8d-7892a71edd3f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:12.199890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c2f796-dd0b-44fe-8d8d-7892a71edd3f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:12.201217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 702/1397] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:20:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:20:12.206236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd74235-92fb-403b-8fd7-9919607fb25e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:12.206435 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd74235-92fb-403b-8fd7-9919607fb25e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:12.206715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd74235-92fb-403b-8fd7-9919607fb25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:12.282121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 696/1398] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:20:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:12.282442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4dd74235-92fb-403b-8fd7-9919607fb25e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:12.283770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4dd74235-92fb-403b-8fd7-9919607fb25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:12.334704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458deb34-662c-44f5-9a5e-29beb669c76f None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:12.335000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458deb34-662c-44f5-9a5e-29beb669c76f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:12.335232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:12.335370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458deb34-662c-44f5-9a5e-29beb669c76f None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:12.336634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 697/1399] 10.222.0.23 () {54 vars in 942 bytes} [Tue Jul 12 13:20:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:20:12.342927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c6357-bb6b-4a0e-af69-0fd153ebea97 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:12.343222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c6357-bb6b-4a0e-af69-0fd153ebea97 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:12.343573 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c6357-bb6b-4a0e-af69-0fd153ebea97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:12.394404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 703/1400] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:20:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:20:12.402127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:12.420844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3b1c6357-bb6b-4a0e-af69-0fd153ebea97 None None] MFA Rules not processed for user `b69264fe173045e08c282171cd7370af`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:12.422093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3b1c6357-bb6b-4a0e-af69-0fd153ebea97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:12.442117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd74235-92fb-403b-8fd7-9919607fb25e None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:12.443318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4dd74235-92fb-403b-8fd7-9919607fb25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:12.530086 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 698/1401] 10.222.0.23 () {58 vars in 1039 bytes} [Tue Jul 12 13:20:12 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:20:12.539834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c6357-bb6b-4a0e-af69-0fd153ebea97 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:12.551516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c6357-bb6b-4a0e-af69-0fd153ebea97 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:12.559390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd74235-92fb-403b-8fd7-9919607fb25e None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:12.626151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:12.664538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:12.670364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17b268-254b-4268-81b5-f9eae31a3173 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:12.671017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17b268-254b-4268-81b5-f9eae31a3173 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:12.671300 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17b268-254b-4268-81b5-f9eae31a3173 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:12.671576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17b268-254b-4268-81b5-f9eae31a3173 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:12.673278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4c17b268-254b-4268-81b5-f9eae31a3173 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:12.688583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784b9546-9a57-4db2-a23a-c0116ec9b453 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:12.689234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9546-9a57-4db2-a23a-c0116ec9b453 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:12.689550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9546-9a57-4db2-a23a-c0116ec9b453 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:12.689829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784b9546-9a57-4db2-a23a-c0116ec9b453 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:12.691737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-784b9546-9a57-4db2-a23a-c0116ec9b453 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:12.755879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17b268-254b-4268-81b5-f9eae31a3173 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:12.757510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17b268-254b-4268-81b5-f9eae31a3173 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:12.766025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 699/1402] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:20:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 13:20:12.787044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b9546-9a57-4db2-a23a-c0116ec9b453 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:12.788656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784b9546-9a57-4db2-a23a-c0116ec9b453 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:12.810755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 704/1403] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:20:14.961081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3098f-581d-482a-b15a-091dfb1f3a11 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:14.961648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3098f-581d-482a-b15a-091dfb1f3a11 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:14.961731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3098f-581d-482a-b15a-091dfb1f3a11 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:14.963513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 700/1404] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:20:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:14.968963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa0480-310b-407e-b20e-3e6efcfa2027 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:14.969448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa0480-310b-407e-b20e-3e6efcfa2027 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:14.969599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfa0480-310b-407e-b20e-3e6efcfa2027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:15.040921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9bfa0480-310b-407e-b20e-3e6efcfa2027 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:15.042726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9bfa0480-310b-407e-b20e-3e6efcfa2027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:15.063953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:15.151815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 705/1405] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:15.197972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3098f-581d-482a-b15a-091dfb1f3a11 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:15.199525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-44d3098f-581d-482a-b15a-091dfb1f3a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:15.374345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:15.385528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3098f-581d-482a-b15a-091dfb1f3a11 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:15.511028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c4c2e6-0659-40ab-8ce1-d1990b91f493 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:15.511593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4c2e6-0659-40ab-8ce1-d1990b91f493 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:15.512088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4c2e6-0659-40ab-8ce1-d1990b91f493 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:15.512362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c4c2e6-0659-40ab-8ce1-d1990b91f493 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:15.514417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-16c4c2e6-0659-40ab-8ce1-d1990b91f493 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:15.618500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4c2e6-0659-40ab-8ce1-d1990b91f493 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:15.620893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c4c2e6-0659-40ab-8ce1-d1990b91f493 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:15.633417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:15.657428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 701/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:17.853528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068dd9e-2569-48ab-aae5-1c387d24b879 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:17.854123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068dd9e-2569-48ab-aae5-1c387d24b879 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:17.854227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6068dd9e-2569-48ab-aae5-1c387d24b879 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:17.856140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 706/1407] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:20:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:17.861596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fbd20-2f4a-4602-9b5a-299b9cde477c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:17.862085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fbd20-2f4a-4602-9b5a-299b9cde477c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:17.862177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4fbd20-2f4a-4602-9b5a-299b9cde477c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:17.936439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0c4fbd20-2f4a-4602-9b5a-299b9cde477c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:17.937693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0c4fbd20-2f4a-4602-9b5a-299b9cde477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:18.056141 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 702/1408] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:18.103033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068dd9e-2569-48ab-aae5-1c387d24b879 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:18.105504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6068dd9e-2569-48ab-aae5-1c387d24b879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:18.274958 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:18.276890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6068dd9e-2569-48ab-aae5-1c387d24b879 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:18.401534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876f139d-31df-49e8-aa8f-4fe53bf6c2da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:18.402141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f139d-31df-49e8-aa8f-4fe53bf6c2da service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:18.402413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f139d-31df-49e8-aa8f-4fe53bf6c2da service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:18.402684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876f139d-31df-49e8-aa8f-4fe53bf6c2da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:18.404369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-876f139d-31df-49e8-aa8f-4fe53bf6c2da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:18.498617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f139d-31df-49e8-aa8f-4fe53bf6c2da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:18.500237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876f139d-31df-49e8-aa8f-4fe53bf6c2da service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:18.522223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 707/1409] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:28.054398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=67 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:28.519651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:35.099281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe172b7-6455-4afc-8f3c-e515e0ed23f0 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:35.099892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe172b7-6455-4afc-8f3c-e515e0ed23f0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:35.099892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe172b7-6455-4afc-8f3c-e515e0ed23f0 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:35.101219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 703/1410] 10.222.0.23 () {54 vars in 918 bytes} [Tue Jul 12 13:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:35.106787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f493f3-21f0-45b1-baf9-a1834b86dca1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:35.107242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f493f3-21f0-45b1-baf9-a1834b86dca1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:35.107357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f493f3-21f0-45b1-baf9-a1834b86dca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:35.180328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-81f493f3-21f0-45b1-baf9-a1834b86dca1 None None] MFA Rules not processed for user `2e97119806774c2482a9886b11ab6f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:35.181566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-81f493f3-21f0-45b1-baf9-a1834b86dca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:35.286264 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 708/1411] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Jul 12 13:20:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:35.300299 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb7cc2-e2b4-4f0c-a173-184b1bdbe47e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:35.300592 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb7cc2-e2b4-4f0c-a173-184b1bdbe47e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:35.300872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb7cc2-e2b4-4f0c-a173-184b1bdbe47e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:35.302254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 704/1412] 10.222.0.23 () {54 vars in 918 bytes} [Tue Jul 12 13:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:35.306918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498a198-0a1b-452c-8332-e16adca56ae1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:35.307370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498a198-0a1b-452c-8332-e16adca56ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:35.307797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498a198-0a1b-452c-8332-e16adca56ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:35.380320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:35.383249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f498a198-0a1b-452c-8332-e16adca56ae1 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:35.385893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f498a198-0a1b-452c-8332-e16adca56ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:35.500340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 709/1413] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Jul 12 13:20:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:37.988866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fe483-9305-4274-8b2b-84f80a8b61b6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:37.989445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fe483-9305-4274-8b2b-84f80a8b61b6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:37.989445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5fe483-9305-4274-8b2b-84f80a8b61b6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:37.990823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:37.996582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa22711-8a79-4c99-a615-803e92aee294 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:37.997016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa22711-8a79-4c99-a615-803e92aee294 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:37.997118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa22711-8a79-4c99-a615-803e92aee294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:38.063658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5aa22711-8a79-4c99-a615-803e92aee294 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:38.064872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5aa22711-8a79-4c99-a615-803e92aee294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:38.118637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:38.160661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:20:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:38.205824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fe483-9305-4274-8b2b-84f80a8b61b6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:38.207389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5e5fe483-9305-4274-8b2b-84f80a8b61b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:38.351968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5fe483-9305-4274-8b2b-84f80a8b61b6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:38.423329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:38.463477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490de8ba-d28b-43ec-b9ef-4a415a29981d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:38.464144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490de8ba-d28b-43ec-b9ef-4a415a29981d service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:38.464471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490de8ba-d28b-43ec-b9ef-4a415a29981d service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:38.464790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490de8ba-d28b-43ec-b9ef-4a415a29981d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:38.466522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-490de8ba-d28b-43ec-b9ef-4a415a29981d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:38.558066 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490de8ba-d28b-43ec-b9ef-4a415a29981d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:38.559927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490de8ba-d28b-43ec-b9ef-4a415a29981d service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:38.578440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 706/1416] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:39.491048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1920e-b674-450d-b371-d0b207c3a41d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:39.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1920e-b674-450d-b371-d0b207c3a41d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:39.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f1920e-b674-450d-b371-d0b207c3a41d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:39.492985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:39.498599 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740ccc-9abd-44ca-b6b6-99c1848faf34 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:39.498999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740ccc-9abd-44ca-b6b6-99c1848faf34 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:39.499110 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3740ccc-9abd-44ca-b6b6-99c1848faf34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:39.573827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f3740ccc-9abd-44ca-b6b6-99c1848faf34 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:39.575057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f3740ccc-9abd-44ca-b6b6-99c1848faf34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:39.600989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:39.669887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:20:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:39.824887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1920e-b674-450d-b371-d0b207c3a41d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:39.826414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e6f1920e-b674-450d-b371-d0b207c3a41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:39.977603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f1920e-b674-450d-b371-d0b207c3a41d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:40.017426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:40.091605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683cd2a-42ba-44af-95d8-21d6cccd67aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:40.092179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683cd2a-42ba-44af-95d8-21d6cccd67aa service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:40.092462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683cd2a-42ba-44af-95d8-21d6cccd67aa service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:40.092736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683cd2a-42ba-44af-95d8-21d6cccd67aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:40.094426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e683cd2a-42ba-44af-95d8-21d6cccd67aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:40.183704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683cd2a-42ba-44af-95d8-21d6cccd67aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:40.185337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e683cd2a-42ba-44af-95d8-21d6cccd67aa service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:40.203913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 712/1419] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:41.084349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008a002-9136-4e22-9bfe-3a81e5def499 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:41.084894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008a002-9136-4e22-9bfe-3a81e5def499 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:41.084894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008a002-9136-4e22-9bfe-3a81e5def499 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:41.086309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:41.091987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a70292-975d-4293-a295-215ef2af1435 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:41.092415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a70292-975d-4293-a295-215ef2af1435 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:41.092580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a70292-975d-4293-a295-215ef2af1435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:41.145254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:41.167788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f6a70292-975d-4293-a295-215ef2af1435 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:41.168993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f6a70292-975d-4293-a295-215ef2af1435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:41.265177 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:20:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:41.424947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008a002-9136-4e22-9bfe-3a81e5def499 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:41.426525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c008a002-9136-4e22-9bfe-3a81e5def499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:41.571134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008a002-9136-4e22-9bfe-3a81e5def499 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:41.583091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:41.684272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b682b6-55a6-4ccc-9360-da0d24799d85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:41.684862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b682b6-55a6-4ccc-9360-da0d24799d85 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:41.685150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b682b6-55a6-4ccc-9360-da0d24799d85 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:41.685460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b682b6-55a6-4ccc-9360-da0d24799d85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:41.687114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-89b682b6-55a6-4ccc-9360-da0d24799d85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:41.774458 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b682b6-55a6-4ccc-9360-da0d24799d85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:41.776563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b682b6-55a6-4ccc-9360-da0d24799d85 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:41.797841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 709/1422] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:42.665296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7db618-9670-41d8-9d66-f205e2709fb3 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:42.666597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7db618-9670-41d8-9d66-f205e2709fb3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:42.666597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7db618-9670-41d8-9d66-f205e2709fb3 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:42.667775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:20:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:42.672935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976562f-b7f1-48f1-b665-6f0b08c4239e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:42.673311 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976562f-b7f1-48f1-b665-6f0b08c4239e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:42.673526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976562f-b7f1-48f1-b665-6f0b08c4239e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:42.691940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:42.749694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9976562f-b7f1-48f1-b665-6f0b08c4239e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:42.750921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9976562f-b7f1-48f1-b665-6f0b08c4239e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:42.849235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:42.999887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7db618-9670-41d8-9d66-f205e2709fb3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:43.001430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7c7db618-9670-41d8-9d66-f205e2709fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:43.110922 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:43.146806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7db618-9670-41d8-9d66-f205e2709fb3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:43.263744 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553fad2f-4102-4beb-8d15-30b9bd6de39b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:43.264340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fad2f-4102-4beb-8d15-30b9bd6de39b service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:43.264710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fad2f-4102-4beb-8d15-30b9bd6de39b service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:43.265223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553fad2f-4102-4beb-8d15-30b9bd6de39b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:43.268071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-553fad2f-4102-4beb-8d15-30b9bd6de39b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:43.345450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:43.361543 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fad2f-4102-4beb-8d15-30b9bd6de39b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:43.363171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553fad2f-4102-4beb-8d15-30b9bd6de39b service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:43.381409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 715/1425] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:44.242626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3322b-4d0a-43ac-be7f-a8a279668b59 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:44.243155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3322b-4d0a-43ac-be7f-a8a279668b59 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:44.243266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee3322b-4d0a-43ac-be7f-a8a279668b59 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:44.245125 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:20:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:44.250120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f372eb-5cb3-4d3a-b37a-0ab9d64f830b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:44.250563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f372eb-5cb3-4d3a-b37a-0ab9d64f830b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:44.250665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f372eb-5cb3-4d3a-b37a-0ab9d64f830b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:44.325893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-55f372eb-5cb3-4d3a-b37a-0ab9d64f830b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:44.327103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-55f372eb-5cb3-4d3a-b37a-0ab9d64f830b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:44.421235 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:20:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:44.572528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3322b-4d0a-43ac-be7f-a8a279668b59 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:44.573988 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7ee3322b-4d0a-43ac-be7f-a8a279668b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:44.656559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:44.719472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee3322b-4d0a-43ac-be7f-a8a279668b59 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:44.829919 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d88276f-5b96-4a67-b776-9d7beb4300ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:44.830495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88276f-5b96-4a67-b776-9d7beb4300ee service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:44.830781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88276f-5b96-4a67-b776-9d7beb4300ee service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:44.831085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d88276f-5b96-4a67-b776-9d7beb4300ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:44.832784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9d88276f-5b96-4a67-b776-9d7beb4300ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:44.879599 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:44.926389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88276f-5b96-4a67-b776-9d7beb4300ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:44.928015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d88276f-5b96-4a67-b776-9d7beb4300ee service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:44.946818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 712/1428] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:45.751448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665068a-7146-4591-a985-4b6682d1120f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:45.751959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665068a-7146-4591-a985-4b6682d1120f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:45.752076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8665068a-7146-4591-a985-4b6682d1120f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:45.753414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:20:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:45.758446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f2d99-a0a7-4867-bfcd-d6d9bc494a50 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:45.758853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f2d99-a0a7-4867-bfcd-d6d9bc494a50 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:45.759000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f2d99-a0a7-4867-bfcd-d6d9bc494a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:45.832923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-111f2d99-a0a7-4867-bfcd-d6d9bc494a50 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:45.834167 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-111f2d99-a0a7-4867-bfcd-d6d9bc494a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:45.929527 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:20:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:46.083434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665068a-7146-4591-a985-4b6682d1120f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:46.084989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8665068a-7146-4591-a985-4b6682d1120f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:46.133740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:46.232633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8665068a-7146-4591-a985-4b6682d1120f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:46.349518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672e82a-4e40-4e40-9734-163a4715070a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:46.350171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e82a-4e40-4e40-9734-163a4715070a service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:46.350455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e82a-4e40-4e40-9734-163a4715070a service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:46.350733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e82a-4e40-4e40-9734-163a4715070a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:46.352387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b672e82a-4e40-4e40-9734-163a4715070a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:46.359254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:46.440490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672e82a-4e40-4e40-9734-163a4715070a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:46.442391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672e82a-4e40-4e40-9734-163a4715070a service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:46.460530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 718/1431] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:47.261995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe97d3-2337-466e-b1af-c2bf50645f8c None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:47.262520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe97d3-2337-466e-b1af-c2bf50645f8c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:47.262627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fe97d3-2337-466e-b1af-c2bf50645f8c None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:47.263914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:20:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:47.268858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde073d8-dcea-40c5-a91c-e06b9e80f2e6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:47.269260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde073d8-dcea-40c5-a91c-e06b9e80f2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:47.269464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde073d8-dcea-40c5-a91c-e06b9e80f2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:47.344671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fde073d8-dcea-40c5-a91c-e06b9e80f2e6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:47.345918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fde073d8-dcea-40c5-a91c-e06b9e80f2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:47.414545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:47.443780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:20:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:47.594623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe97d3-2337-466e-b1af-c2bf50645f8c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:47.596060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-95fe97d3-2337-466e-b1af-c2bf50645f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:47.613162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:47.740879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fe97d3-2337-466e-b1af-c2bf50645f8c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:47.826839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:47.852469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b35606-2c28-4580-aa82-1a9008903242 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:47.853081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b35606-2c28-4580-aa82-1a9008903242 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:47.853366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b35606-2c28-4580-aa82-1a9008903242 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:47.853673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b35606-2c28-4580-aa82-1a9008903242 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:47.855332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b3b35606-2c28-4580-aa82-1a9008903242 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:47.944340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b35606-2c28-4580-aa82-1a9008903242 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:47.945992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b35606-2c28-4580-aa82-1a9008903242 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:47.964765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 715/1434] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:48.774088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562038b5-d06d-49aa-93a5-ec34b9ec3618 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:48.775339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562038b5-d06d-49aa-93a5-ec34b9ec3618 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:48.775339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562038b5-d06d-49aa-93a5-ec34b9ec3618 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:48.776783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:20:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:48.781878 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd704a5-59c5-4b5c-9d67-afd046041e92 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:48.782319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd704a5-59c5-4b5c-9d67-afd046041e92 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:48.782423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd704a5-59c5-4b5c-9d67-afd046041e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:48.856432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-acd704a5-59c5-4b5c-9d67-afd046041e92 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:48.857672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-acd704a5-59c5-4b5c-9d67-afd046041e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:48.898415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:48.956488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:20:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:49.107473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562038b5-d06d-49aa-93a5-ec34b9ec3618 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:49.108989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-562038b5-d06d-49aa-93a5-ec34b9ec3618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:49.254943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562038b5-d06d-49aa-93a5-ec34b9ec3618 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:49.308579 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:49.370995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6529e627-3e3c-4046-a58e-c40e7be26b9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:49.371885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529e627-3e3c-4046-a58e-c40e7be26b9c service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:49.372225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529e627-3e3c-4046-a58e-c40e7be26b9c service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:49.372513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6529e627-3e3c-4046-a58e-c40e7be26b9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:49.374683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6529e627-3e3c-4046-a58e-c40e7be26b9c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:49.463073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529e627-3e3c-4046-a58e-c40e7be26b9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:49.464667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6529e627-3e3c-4046-a58e-c40e7be26b9c service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:49.482467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 721/1437] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:50.291408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f14ec4-f9d0-4d4d-9637-380550658817 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:50.291937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f14ec4-f9d0-4d4d-9637-380550658817 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:50.292047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f14ec4-f9d0-4d4d-9637-380550658817 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:50.293328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:20:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:50.298182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410bcd7-3981-48cf-9c84-d7f42de2af4d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:50.298601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410bcd7-3981-48cf-9c84-d7f42de2af4d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:50.298749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410bcd7-3981-48cf-9c84-d7f42de2af4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:50.372171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4410bcd7-3981-48cf-9c84-d7f42de2af4d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:50.373393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4410bcd7-3981-48cf-9c84-d7f42de2af4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:50.375382 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:50.468092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:20:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:50.618791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f14ec4-f9d0-4d4d-9637-380550658817 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:50.620316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c3f14ec4-f9d0-4d4d-9637-380550658817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:50.764466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f14ec4-f9d0-4d4d-9637-380550658817 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:50.789688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:50.876088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a91f442-ae5a-43dc-96fe-79767fbfd686 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:50.876655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91f442-ae5a-43dc-96fe-79767fbfd686 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:50.876939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91f442-ae5a-43dc-96fe-79767fbfd686 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:50.877210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a91f442-ae5a-43dc-96fe-79767fbfd686 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:50.878878 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0a91f442-ae5a-43dc-96fe-79767fbfd686 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:50.967289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91f442-ae5a-43dc-96fe-79767fbfd686 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:50.968873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a91f442-ae5a-43dc-96fe-79767fbfd686 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:50.986957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 718/1440] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:52.937478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef98aa-8098-4755-81bc-1a5e516b2001 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:52.938048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef98aa-8098-4755-81bc-1a5e516b2001 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:52.938048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef98aa-8098-4755-81bc-1a5e516b2001 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:52.939392 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:52.944745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91cf2f-9070-43ff-ba92-84c63e9db88f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:52.945143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91cf2f-9070-43ff-ba92-84c63e9db88f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:52.945296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c91cf2f-9070-43ff-ba92-84c63e9db88f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:52.983524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:53.008489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5c91cf2f-9070-43ff-ba92-84c63e9db88f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:53.009859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5c91cf2f-9070-43ff-ba92-84c63e9db88f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:53.103473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:20:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:53.142106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef98aa-8098-4755-81bc-1a5e516b2001 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:53.143630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-43ef98aa-8098-4755-81bc-1a5e516b2001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:53.245814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef98aa-8098-4755-81bc-1a5e516b2001 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:53.277500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:53.320155 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614e09a2-3bbe-4118-ae89-a6a83217bf4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:53.320816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e09a2-3bbe-4118-ae89-a6a83217bf4b service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:53.321160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e09a2-3bbe-4118-ae89-a6a83217bf4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:53.321475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614e09a2-3bbe-4118-ae89-a6a83217bf4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:53.323142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-614e09a2-3bbe-4118-ae89-a6a83217bf4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:53.434331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e09a2-3bbe-4118-ae89-a6a83217bf4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:53.435768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614e09a2-3bbe-4118-ae89-a6a83217bf4b service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:53.453730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 724/1443] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:55.656393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b4286-d8e0-44cd-970e-0571e5f7dc6a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:55.656764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b4286-d8e0-44cd-970e-0571e5f7dc6a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:55.656878 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b4286-d8e0-44cd-970e-0571e5f7dc6a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:55.658225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:55.663517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3eac0-25b7-4eb7-b195-600efab2f285 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:55.663944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3eac0-25b7-4eb7-b195-600efab2f285 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:55.664045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3eac0-25b7-4eb7-b195-600efab2f285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:55.728059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5cb3eac0-25b7-4eb7-b195-600efab2f285 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:55.729277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5cb3eac0-25b7-4eb7-b195-600efab2f285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:55.741702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:55.824247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:20:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:20:55.862142 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b4286-d8e0-44cd-970e-0571e5f7dc6a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:55.863683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4b0b4286-d8e0-44cd-970e-0571e5f7dc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:55.967594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b4286-d8e0-44cd-970e-0571e5f7dc6a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:55.981202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:56.043077 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50f9b2-12ff-47d4-b86b-f9852abd7e37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:56.043726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50f9b2-12ff-47d4-b86b-f9852abd7e37 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:56.044016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50f9b2-12ff-47d4-b86b-f9852abd7e37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:56.044291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50f9b2-12ff-47d4-b86b-f9852abd7e37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:56.045987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7f50f9b2-12ff-47d4-b86b-f9852abd7e37 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:56.153613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50f9b2-12ff-47d4-b86b-f9852abd7e37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:56.155055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50f9b2-12ff-47d4-b86b-f9852abd7e37 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:56.172538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 721/1446] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:20:58.516938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918f7e5-30b8-4e8c-85cd-57f55028843f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:58.517506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918f7e5-30b8-4e8c-85cd-57f55028843f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:58.517506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f918f7e5-30b8-4e8c-85cd-57f55028843f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:58.518887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:20:58.524096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf6a60-777a-4c9d-90bb-bce363716c6b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:58.524515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf6a60-777a-4c9d-90bb-bce363716c6b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:58.524693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf6a60-777a-4c9d-90bb-bce363716c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:58.576934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:58.594777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-66cf6a60-777a-4c9d-90bb-bce363716c6b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:20:58.595998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-66cf6a60-777a-4c9d-90bb-bce363716c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:58.691494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:20:58.729838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918f7e5-30b8-4e8c-85cd-57f55028843f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:20:58.731366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f918f7e5-30b8-4e8c-85cd-57f55028843f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:58.833087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f918f7e5-30b8-4e8c-85cd-57f55028843f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:20:58.872289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:20:58.905272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60274c0-2b30-43ac-9c74-5b6b0eb4a9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:20:58.905944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60274c0-2b30-43ac-9c74-5b6b0eb4a9fa service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:20:58.906229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60274c0-2b30-43ac-9c74-5b6b0eb4a9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:20:58.906505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60274c0-2b30-43ac-9c74-5b6b0eb4a9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:20:58.908146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e60274c0-2b30-43ac-9c74-5b6b0eb4a9fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:20:59.017570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60274c0-2b30-43ac-9c74-5b6b0eb4a9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:20:59.019008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60274c0-2b30-43ac-9c74-5b6b0eb4a9fa service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:20:59.036799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 727/1449] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:20:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:01.225565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89499878-ca88-40c0-b1ba-6f5d6e8b4262 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:01.226178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89499878-ca88-40c0-b1ba-6f5d6e8b4262 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:01.226178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89499878-ca88-40c0-b1ba-6f5d6e8b4262 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:01.228039 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:01.233671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6f7a6-de94-46e5-81e2-0214533662d2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:01.234081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6f7a6-de94-46e5-81e2-0214533662d2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:01.234231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d6f7a6-de94-46e5-81e2-0214533662d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:01.299262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-22d6f7a6-de94-46e5-81e2-0214533662d2 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:01.300466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-22d6f7a6-de94-46e5-81e2-0214533662d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:01.321415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:01.396071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:01.438985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89499878-ca88-40c0-b1ba-6f5d6e8b4262 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:01.440541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-89499878-ca88-40c0-b1ba-6f5d6e8b4262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:01.543046 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89499878-ca88-40c0-b1ba-6f5d6e8b4262 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:01.562740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:01.618853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83694f7-9f61-4090-8f44-000cb3ef6860 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:01.619500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83694f7-9f61-4090-8f44-000cb3ef6860 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:01.619782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83694f7-9f61-4090-8f44-000cb3ef6860 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:01.620055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83694f7-9f61-4090-8f44-000cb3ef6860 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:01.621748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f83694f7-9f61-4090-8f44-000cb3ef6860 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:01.741291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83694f7-9f61-4090-8f44-000cb3ef6860 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:01.743344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83694f7-9f61-4090-8f44-000cb3ef6860 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:01.761668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 724/1452] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:03.923687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3005524-5740-49f9-8a46-5611706d47d5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:03.924219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3005524-5740-49f9-8a46-5611706d47d5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:03.924219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3005524-5740-49f9-8a46-5611706d47d5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:03.925618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:03.931476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12967101-7357-472f-a4f4-3498369089fb None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:03.931914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12967101-7357-472f-a4f4-3498369089fb None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:03.932024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12967101-7357-472f-a4f4-3498369089fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:03.994616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:03.997349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-12967101-7357-472f-a4f4-3498369089fb None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:03.999230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-12967101-7357-472f-a4f4-3498369089fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:04.093596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:21:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:04.131079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3005524-5740-49f9-8a46-5611706d47d5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:04.133657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a3005524-5740-49f9-8a46-5611706d47d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:04.235306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3005524-5740-49f9-8a46-5611706d47d5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:04.284023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:04.309660 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d916d-572b-41fd-9048-e2562e587756 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:04.310310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d916d-572b-41fd-9048-e2562e587756 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:04.310599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d916d-572b-41fd-9048-e2562e587756 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:04.310875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d916d-572b-41fd-9048-e2562e587756 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:04.312529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6e8d916d-572b-41fd-9048-e2562e587756 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:04.421461 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d916d-572b-41fd-9048-e2562e587756 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:04.422912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d916d-572b-41fd-9048-e2562e587756 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:04.441520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 730/1455] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:06.509150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955975e1-20c5-46b9-b3e1-3d037c7d5a75 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:06.509716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955975e1-20c5-46b9-b3e1-3d037c7d5a75 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:06.509716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955975e1-20c5-46b9-b3e1-3d037c7d5a75 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:06.511011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:21:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:06.516317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f563b87-393e-4cb2-a91b-b482e10add8b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:06.516734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f563b87-393e-4cb2-a91b-b482e10add8b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:06.516884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f563b87-393e-4cb2-a91b-b482e10add8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:06.581174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1f563b87-393e-4cb2-a91b-b482e10add8b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:06.582425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1f563b87-393e-4cb2-a91b-b482e10add8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:06.614013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:06.682198 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:21:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:06.720227 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955975e1-20c5-46b9-b3e1-3d037c7d5a75 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:06.721835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-955975e1-20c5-46b9-b3e1-3d037c7d5a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:06.824020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955975e1-20c5-46b9-b3e1-3d037c7d5a75 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:06.851469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:06.899375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a6fb6e-ea09-4035-9e80-d13d1fa7f542 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:06.900022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6fb6e-ea09-4035-9e80-d13d1fa7f542 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:06.900305 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6fb6e-ea09-4035-9e80-d13d1fa7f542 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:06.900578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a6fb6e-ea09-4035-9e80-d13d1fa7f542 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:06.902253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-41a6fb6e-ea09-4035-9e80-d13d1fa7f542 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:07.011568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6fb6e-ea09-4035-9e80-d13d1fa7f542 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:07.013007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a6fb6e-ea09-4035-9e80-d13d1fa7f542 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:07.031546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 727/1458] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:09.096837 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4b37a-0af9-4273-b3a0-50a6bf2f0df5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:09.097446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4b37a-0af9-4273-b3a0-50a6bf2f0df5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:09.097446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd4b37a-0af9-4273-b3a0-50a6bf2f0df5 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:09.098858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:21:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:09.104224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c1a53-5d58-4d9a-8030-b29aced3e9d4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:09.104650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c1a53-5d58-4d9a-8030-b29aced3e9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:09.104786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c1a53-5d58-4d9a-8030-b29aced3e9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:09.170267 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-043c1a53-5d58-4d9a-8030-b29aced3e9d4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:09.171460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-043c1a53-5d58-4d9a-8030-b29aced3e9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:09.179987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:09.266657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:21:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:09.303632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4b37a-0af9-4273-b3a0-50a6bf2f0df5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:09.305944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2bd4b37a-0af9-4273-b3a0-50a6bf2f0df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:09.409779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd4b37a-0af9-4273-b3a0-50a6bf2f0df5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:09.472779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:09.490721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7761510f-d76e-4aaa-a81d-69081f552483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:09.491372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761510f-d76e-4aaa-a81d-69081f552483 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:09.491659 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761510f-d76e-4aaa-a81d-69081f552483 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:09.491945 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7761510f-d76e-4aaa-a81d-69081f552483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:09.493760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7761510f-d76e-4aaa-a81d-69081f552483 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:09.600814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761510f-d76e-4aaa-a81d-69081f552483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:09.602326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7761510f-d76e-4aaa-a81d-69081f552483 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:09.619801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 733/1461] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:11.670075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3c0aa-e63e-4627-afd6-3a4ec8ce16c0 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:11.670497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3c0aa-e63e-4627-afd6-3a4ec8ce16c0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:11.670646 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e3c0aa-e63e-4627-afd6-3a4ec8ce16c0 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:11.672015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:11.677289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403843f-2e86-422c-8dff-0ed711dc0408 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:11.677718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403843f-2e86-422c-8dff-0ed711dc0408 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:11.677904 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f403843f-2e86-422c-8dff-0ed711dc0408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:11.744200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f403843f-2e86-422c-8dff-0ed711dc0408 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:11.745448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f403843f-2e86-422c-8dff-0ed711dc0408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:11.783705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:11.841888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:21:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:11.878345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3c0aa-e63e-4627-afd6-3a4ec8ce16c0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:11.879916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-58e3c0aa-e63e-4627-afd6-3a4ec8ce16c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:11.984068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e3c0aa-e63e-4627-afd6-3a4ec8ce16c0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:12.021488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:12.057950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae62e2e-5fcf-40d8-a7bd-981580343914 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:12.058602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae62e2e-5fcf-40d8-a7bd-981580343914 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:12.058888 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae62e2e-5fcf-40d8-a7bd-981580343914 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:12.059165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae62e2e-5fcf-40d8-a7bd-981580343914 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:12.060805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9ae62e2e-5fcf-40d8-a7bd-981580343914 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:12.168207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae62e2e-5fcf-40d8-a7bd-981580343914 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:12.169677 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae62e2e-5fcf-40d8-a7bd-981580343914 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:12.187777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 730/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:14.239042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceb8d1-5d21-413a-8f2a-449c80985b94 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:14.239604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceb8d1-5d21-413a-8f2a-449c80985b94 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:14.239604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ceb8d1-5d21-413a-8f2a-449c80985b94 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:14.240997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:14.246383 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f102d6-4c7f-4642-a44f-32a952cd4db5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:14.246797 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f102d6-4c7f-4642-a44f-32a952cd4db5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:14.246958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f102d6-4c7f-4642-a44f-32a952cd4db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:14.311428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d8f102d6-4c7f-4642-a44f-32a952cd4db5 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:14.312638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d8f102d6-4c7f-4642-a44f-32a952cd4db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:14.331364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:14.410064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:21:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:14.447500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceb8d1-5d21-413a-8f2a-449c80985b94 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:14.449052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e8ceb8d1-5d21-413a-8f2a-449c80985b94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:14.553453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ceb8d1-5d21-413a-8f2a-449c80985b94 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:14.619752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:14.627198 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85696330-5e92-428a-a2b5-20a971c220f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:14.627853 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85696330-5e92-428a-a2b5-20a971c220f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:14.628136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85696330-5e92-428a-a2b5-20a971c220f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:14.628418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85696330-5e92-428a-a2b5-20a971c220f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:14.630091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-85696330-5e92-428a-a2b5-20a971c220f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:14.738475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85696330-5e92-428a-a2b5-20a971c220f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:14.739920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85696330-5e92-428a-a2b5-20a971c220f5 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:14.760062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 736/1467] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:16.812303 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aa47e-0b32-4585-8758-1eae895677d6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:16.812811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aa47e-0b32-4585-8758-1eae895677d6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:16.812811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5aa47e-0b32-4585-8758-1eae895677d6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:16.814184 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:21:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:16.820639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717d27e-695e-45dd-8d4e-aba5c8474677 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:16.821154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717d27e-695e-45dd-8d4e-aba5c8474677 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:16.821286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7717d27e-695e-45dd-8d4e-aba5c8474677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:16.885730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7717d27e-695e-45dd-8d4e-aba5c8474677 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:16.886942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7717d27e-695e-45dd-8d4e-aba5c8474677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:16.935632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:16.983912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:21:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:17.019985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5aa47e-0b32-4585-8758-1eae895677d6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:17.021586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-db5aa47e-0b32-4585-8758-1eae895677d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:17.122525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5aa47e-0b32-4585-8758-1eae895677d6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:17.168743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:17.195736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd8d6c-3088-4e7a-a77a-4ca73899ca31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:17.196399 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd8d6c-3088-4e7a-a77a-4ca73899ca31 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:17.196686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd8d6c-3088-4e7a-a77a-4ca73899ca31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:17.196963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd8d6c-3088-4e7a-a77a-4ca73899ca31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:17.198648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-10fd8d6c-3088-4e7a-a77a-4ca73899ca31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:17.308727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd8d6c-3088-4e7a-a77a-4ca73899ca31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:17.310206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd8d6c-3088-4e7a-a77a-4ca73899ca31 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:17.329282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 733/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:26.982057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=22 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:27.326831 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=66 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:37.041466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a925d-2b3f-430c-a9b1-7d47c54f0bf7 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:37.042035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a925d-2b3f-430c-a9b1-7d47c54f0bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:37.042035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95a925d-2b3f-430c-a9b1-7d47c54f0bf7 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:37.043409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 738/1471] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:37.048673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69558e1-e05f-461b-b825-636b9d491824 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:37.049116 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69558e1-e05f-461b-b825-636b9d491824 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:37.049228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69558e1-e05f-461b-b825-636b9d491824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:37.114478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c69558e1-e05f-461b-b825-636b9d491824 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:37.116333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c69558e1-e05f-461b-b825-636b9d491824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:37.215321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 734/1472] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:21:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:21:37.299282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353acf78-2a8e-44fa-8453-35586a0ca49d None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:37.299571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353acf78-2a8e-44fa-8453-35586a0ca49d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:37.299848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353acf78-2a8e-44fa-8453-35586a0ca49d None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:37.301211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 739/1473] 10.222.0.23 () {54 vars in 962 bytes} [Tue Jul 12 13:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:37.306668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3230-97ec-405c-8549-3f92201d7770 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:37.307050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3230-97ec-405c-8549-3f92201d7770 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:37.307332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3230-97ec-405c-8549-3f92201d7770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:37.382570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fd9b3230-97ec-405c-8549-3f92201d7770 None None] MFA Rules not processed for user `0ff10db6937b4f2b889c30dd5fad8111`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:37.383786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fd9b3230-97ec-405c-8549-3f92201d7770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:37.404604 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:37.459811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 735/1474] 10.222.0.23 () {58 vars in 1059 bytes} [Tue Jul 12 13:21:37 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:37.467097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353acf78-2a8e-44fa-8453-35586a0ca49d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:37.468661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-353acf78-2a8e-44fa-8453-35586a0ca49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:37.574108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353acf78-2a8e-44fa-8453-35586a0ca49d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:37.648590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65758e6-bf21-4ac5-a3f6-35974fe31b62 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ff10db6937b4f2b889c30dd5fad8111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:37.649238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65758e6-bf21-4ac5-a3f6-35974fe31b62 service glance] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:37.649565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65758e6-bf21-4ac5-a3f6-35974fe31b62 service glance] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:37.649839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65758e6-bf21-4ac5-a3f6-35974fe31b62 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:37.651485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f65758e6-bf21-4ac5-a3f6-35974fe31b62 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:37.705618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:37.762164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65758e6-bf21-4ac5-a3f6-35974fe31b62 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:37.763612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65758e6-bf21-4ac5-a3f6-35974fe31b62 service glance] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:37.782431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 740/1475] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:39.891882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257ab89-9428-430c-9b7e-a31ee23b5d39 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:39.892421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257ab89-9428-430c-9b7e-a31ee23b5d39 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:39.892538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f257ab89-9428-430c-9b7e-a31ee23b5d39 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:39.893886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 736/1476] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:39.899494 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72a240-5f14-4753-b673-47dc19051fac None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:39.899937 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72a240-5f14-4753-b673-47dc19051fac None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:39.900087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec72a240-5f14-4753-b673-47dc19051fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:39.964477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ec72a240-5f14-4753-b673-47dc19051fac None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:39.965725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ec72a240-5f14-4753-b673-47dc19051fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:40.061062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 741/1477] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:40.085251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257ab89-9428-430c-9b7e-a31ee23b5d39 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:40.086831 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f257ab89-9428-430c-9b7e-a31ee23b5d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:40.220206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f257ab89-9428-430c-9b7e-a31ee23b5d39 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:40.270161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:40.317633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abba603-2a92-4d43-8a96-3edba7285e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:40.319113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba603-2a92-4d43-8a96-3edba7285e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:40.319398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba603-2a92-4d43-8a96-3edba7285e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:40.319675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abba603-2a92-4d43-8a96-3edba7285e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:40.320412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abba603-2a92-4d43-8a96-3edba7285e3d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:40.322253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-6abba603-2a92-4d43-8a96-3edba7285e3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:21:40.322253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 12 13:21:40.322253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: Jul 12 13:21:40.323406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abba603-2a92-4d43-8a96-3edba7285e3d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:40.342629 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 737/1478] 10.222.0.23 () {56 vars in 1108 bytes} [Tue Jul 12 13:21:40 2022] GET /identity/v3/projects => generated 2553 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:21:42.408659 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043db43-d48a-44d7-b513-3692119187ce None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:42.409240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043db43-d48a-44d7-b513-3692119187ce None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:42.409240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b043db43-d48a-44d7-b513-3692119187ce None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:42.410691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 742/1479] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:42.417083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a0b04-6e7d-49c1-b783-d0c7bc6e2138 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:42.417499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a0b04-6e7d-49c1-b783-d0c7bc6e2138 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:42.417690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a0b04-6e7d-49c1-b783-d0c7bc6e2138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:42.483589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-645a0b04-6e7d-49c1-b783-d0c7bc6e2138 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:42.484801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-645a0b04-6e7d-49c1-b783-d0c7bc6e2138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:42.582269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 738/1480] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:21:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:42.625707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043db43-d48a-44d7-b513-3692119187ce None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:42.627378 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b043db43-d48a-44d7-b513-3692119187ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:42.637355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:42.769991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b043db43-d48a-44d7-b513-3692119187ce None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:42.851319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:42.883852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd8b701-1136-4f2a-a7e3-aa50527f7ad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:42.884498 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8b701-1136-4f2a-a7e3-aa50527f7ad0 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:42.885054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8b701-1136-4f2a-a7e3-aa50527f7ad0 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:42.885337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd8b701-1136-4f2a-a7e3-aa50527f7ad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:42.887013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3fd8b701-1136-4f2a-a7e3-aa50527f7ad0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:42.977940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8b701-1136-4f2a-a7e3-aa50527f7ad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:42.979594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd8b701-1136-4f2a-a7e3-aa50527f7ad0 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:42.997899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 743/1481] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:45.285693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe64c6e-d666-477c-ba74-1b582982c2be None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:45.286186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe64c6e-d666-477c-ba74-1b582982c2be None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:45.286295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe64c6e-d666-477c-ba74-1b582982c2be None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:45.288225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 739/1482] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:21:45.293444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d629-71f7-412c-82c2-a6b4914f3333 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:45.293879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d629-71f7-412c-82c2-a6b4914f3333 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:45.293983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0261d629-71f7-412c-82c2-a6b4914f3333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:45.357554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0261d629-71f7-412c-82c2-a6b4914f3333 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:45.358760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0261d629-71f7-412c-82c2-a6b4914f3333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:45.404209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:45.455040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 744/1483] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:21:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:21:45.491618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe64c6e-d666-477c-ba74-1b582982c2be None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:45.493192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1fe64c6e-d666-477c-ba74-1b582982c2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:45.516697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:45.601244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe64c6e-d666-477c-ba74-1b582982c2be None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:45.672632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe26791f-7c60-4aec-b90d-3fe0e710cf4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:45.673277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26791f-7c60-4aec-b90d-3fe0e710cf4e service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:45.673588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26791f-7c60-4aec-b90d-3fe0e710cf4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:45.673863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe26791f-7c60-4aec-b90d-3fe0e710cf4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:45.675506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fe26791f-7c60-4aec-b90d-3fe0e710cf4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:45.748610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:45.784097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26791f-7c60-4aec-b90d-3fe0e710cf4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:45.785504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe26791f-7c60-4aec-b90d-3fe0e710cf4e service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:45.803858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 740/1484] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:47.892836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d053985-91d0-4516-8a2f-3262e8f8f520 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:47.893362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d053985-91d0-4516-8a2f-3262e8f8f520 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:47.893510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d053985-91d0-4516-8a2f-3262e8f8f520 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:47.894799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 745/1485] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:21:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:47.900761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039cecf-ce72-409c-98ca-75f70aae581e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:47.901225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039cecf-ce72-409c-98ca-75f70aae581e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:47.901323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6039cecf-ce72-409c-98ca-75f70aae581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:47.965400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6039cecf-ce72-409c-98ca-75f70aae581e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:47.966633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6039cecf-ce72-409c-98ca-75f70aae581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:48.067817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 741/1486] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:21:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:48.105926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d053985-91d0-4516-8a2f-3262e8f8f520 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:48.107456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2d053985-91d0-4516-8a2f-3262e8f8f520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:48.212601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d053985-91d0-4516-8a2f-3262e8f8f520 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:48.286481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:48.288683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827f2c9-cb1c-42b9-87fd-9f0a9b3f4489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:48.289350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f2c9-cb1c-42b9-87fd-9f0a9b3f4489 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:48.289852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f2c9-cb1c-42b9-87fd-9f0a9b3f4489 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:48.290301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827f2c9-cb1c-42b9-87fd-9f0a9b3f4489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:48.292205 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8827f2c9-cb1c-42b9-87fd-9f0a9b3f4489 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:48.400896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827f2c9-cb1c-42b9-87fd-9f0a9b3f4489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:48.402363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8827f2c9-cb1c-42b9-87fd-9f0a9b3f4489 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:48.420930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 746/1487] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:50.653943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db483c-7b71-4eda-bf02-8964d88683e6 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:50.654451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db483c-7b71-4eda-bf02-8964d88683e6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:50.654563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db483c-7b71-4eda-bf02-8964d88683e6 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:50.655864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 742/1488] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:50.660929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3982766-f9c5-4ec0-9b13-0ec1c2c51e49 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:50.661403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3982766-f9c5-4ec0-9b13-0ec1c2c51e49 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:50.662071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3982766-f9c5-4ec0-9b13-0ec1c2c51e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:50.727045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a3982766-f9c5-4ec0-9b13-0ec1c2c51e49 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:50.728257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a3982766-f9c5-4ec0-9b13-0ec1c2c51e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:50.779274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:50.823767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 747/1489] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:50.863627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db483c-7b71-4eda-bf02-8964d88683e6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:50.865987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-19db483c-7b71-4eda-bf02-8964d88683e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:50.901385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:50.970865 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db483c-7b71-4eda-bf02-8964d88683e6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:51.043219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f77ded-42b3-4c5d-8b66-e8f39b810c58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:51.043869 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77ded-42b3-4c5d-8b66-e8f39b810c58 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:51.044152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77ded-42b3-4c5d-8b66-e8f39b810c58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:51.044426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f77ded-42b3-4c5d-8b66-e8f39b810c58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:51.046117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-83f77ded-42b3-4c5d-8b66-e8f39b810c58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:51.127062 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:51.158729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f77ded-42b3-4c5d-8b66-e8f39b810c58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:51.160174 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f77ded-42b3-4c5d-8b66-e8f39b810c58 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:51.178804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 743/1490] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:21:53.383293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a24f4-38d3-4a37-8b6c-7d32e9cd369a None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:53.383867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a24f4-38d3-4a37-8b6c-7d32e9cd369a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:53.383867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09a24f4-38d3-4a37-8b6c-7d32e9cd369a None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:53.385262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 748/1491] 10.222.0.23 () {54 vars in 926 bytes} [Tue Jul 12 13:21:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:21:53.390414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669fd70-af42-408b-975e-0b66d40c980e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:53.390806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669fd70-af42-408b-975e-0b66d40c980e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:53.390976 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1669fd70-af42-408b-975e-0b66d40c980e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:53.455574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1669fd70-af42-408b-975e-0b66d40c980e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:21:53.456791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1669fd70-af42-408b-975e-0b66d40c980e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:53.554351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 744/1492] 10.222.0.23 () {58 vars in 1023 bytes} [Tue Jul 12 13:21:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:21:53.593898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a24f4-38d3-4a37-8b6c-7d32e9cd369a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:21:53.595439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c09a24f4-38d3-4a37-8b6c-7d32e9cd369a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:53.698708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09a24f4-38d3-4a37-8b6c-7d32e9cd369a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:21:53.770181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa3ddf5-baca-4c4e-a24c-c39cc5c70fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:21:53.771663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3ddf5-baca-4c4e-a24c-c39cc5c70fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:21:53.771852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3ddf5-baca-4c4e-a24c-c39cc5c70fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:21:53.772130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa3ddf5-baca-4c4e-a24c-c39cc5c70fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:21:53.773820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5fa3ddf5-baca-4c4e-a24c-c39cc5c70fa0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:21:53.782910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:21:53.888243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3ddf5-baca-4c4e-a24c-c39cc5c70fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:21:53.889717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa3ddf5-baca-4c4e-a24c-c39cc5c70fa0 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:21:53.907251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 749/1493] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:22:03.552367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=82 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:22:03.905055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=50 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:26.345091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239a45-7b47-4ebc-af5c-979d7b74e87d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:26.345701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239a45-7b47-4ebc-af5c-979d7b74e87d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:26.345701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de239a45-7b47-4ebc-af5c-979d7b74e87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:26.418826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-de239a45-7b47-4ebc-af5c-979d7b74e87d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:23:26.420047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-de239a45-7b47-4ebc-af5c-979d7b74e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:26.517100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 745/1494] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:23:26.525671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc7183-39cd-4811-a77e-e751ba5f23c9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:26.526158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc7183-39cd-4811-a77e-e751ba5f23c9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:26.526260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acc7183-39cd-4811-a77e-e751ba5f23c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:26.720863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5acc7183-39cd-4811-a77e-e751ba5f23c9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:23:26.722118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5acc7183-39cd-4811-a77e-e751ba5f23c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:26.817186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 750/1495] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:23:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:23:26.821926 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239a45-7b47-4ebc-af5c-979d7b74e87d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:26.823156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-de239a45-7b47-4ebc-af5c-979d7b74e87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:26.834965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de239a45-7b47-4ebc-af5c-979d7b74e87d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:26.894867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:26.927568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:26.928223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:26.928504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:26.928778 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:26.929528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:23:26.929836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:26.930867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:23:26.930867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 12 13:23:26.930867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: Jul 12 13:23:26.931772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:26.944478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 746/1496] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:23:26.952708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc7183-39cd-4811-a77e-e751ba5f23c9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:26.963041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acc7183-39cd-4811-a77e-e751ba5f23c9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:27.022702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:27.057092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:27.058228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:27.058518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:27.058800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:27.059501 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:27.061233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:23:27.061233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 12 13:23:27.061233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: Jul 12 13:23:27.062456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:27.117755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 751/1497] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:23:26 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:23:27.121897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:27.123481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:27.156048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b047a-e30d-4546-a967-566420543f73 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:27.248747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:27.248932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:27.249211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:27.249516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:27.250198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:27.251213 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:23:27.251213 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 12 13:23:27.251213 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: Jul 12 13:23:27.252410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:27.293149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 747/1498] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:23:27 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:23:27.297243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:27.298780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:27.360372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e7425-a7c5-4210-bcb6-7a31367372b5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:27.441244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:27.451765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:27.452428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:27.452710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:27.452984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:27.453762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:27.455145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:23:27.455145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 12 13:23:27.455145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: Jul 12 13:23:27.456028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:27.469776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 752/1499] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:23:27.474850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:27.476746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:27.489060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b13fa6-b89c-4b35-84cb-255f7aafbcaf admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:27.513579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:27.585105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5591fac3-8518-47e2-91fe-d7e564d8a50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:27.585810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591fac3-8518-47e2-91fe-d7e564d8a50c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:27.586100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591fac3-8518-47e2-91fe-d7e564d8a50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:27.586379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5591fac3-8518-47e2-91fe-d7e564d8a50c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:27.587116 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591fac3-8518-47e2-91fe-d7e564d8a50c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:27.588130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-5591fac3-8518-47e2-91fe-d7e564d8a50c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:23:27.588130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 12 13:23:27.588130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: Jul 12 13:23:27.589043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5591fac3-8518-47e2-91fe-d7e564d8a50c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:27.602296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 748/1500] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:23:27 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:23:27.606573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:27.608152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:27.641525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45481f14-24b5-40d9-b83a-10a7d0fefbc6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:27.732652 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcbad2-dc5a-4f79-90b4-a77c55e289a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:27.733565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcbad2-dc5a-4f79-90b4-a77c55e289a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:27.733846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcbad2-dc5a-4f79-90b4-a77c55e289a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:27.734117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcbad2-dc5a-4f79-90b4-a77c55e289a6 admin admin] PATH_INFO: `/v3/projects/5fc69c3343de413692f464989deb5b80/users/90bade92657d4302b2fcb8969112d13c/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:27.755968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcbad2-dc5a-4f79-90b4-a77c55e289a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fc69c3343de413692f464989deb5b80, user_id=90bade92657d4302b2fcb8969112d13c, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:27.757091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-73fcbad2-dc5a-4f79-90b4-a77c55e289a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:23:27.757091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 12 13:23:27.757091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: Jul 12 13:23:27.759067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcbad2-dc5a-4f79-90b4-a77c55e289a6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:27.783230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 753/1501] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:23:27 2022] PUT /identity/v3/projects/5fc69c3343de413692f464989deb5b80/users/90bade92657d4302b2fcb8969112d13c/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:23:27.795835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ab8f-7208-4844-b052-04eb353d4fcf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:27.796130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ab8f-7208-4844-b052-04eb353d4fcf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:27.796412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ab8f-7208-4844-b052-04eb353d4fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:27.851467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e376ab8f-7208-4844-b052-04eb353d4fcf None None] MFA Rules not processed for user `90bade92657d4302b2fcb8969112d13c`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:23:27.852708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e376ab8f-7208-4844-b052-04eb353d4fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:27.929471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 749/1502] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:23:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:23:27.942677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e48fb-cafc-4007-a340-2da1bd012b36 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:27.942971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e48fb-cafc-4007-a340-2da1bd012b36 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:27.943249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e48fb-cafc-4007-a340-2da1bd012b36 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:27.944641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 754/1503] 10.222.0.23 () {54 vars in 963 bytes} [Tue Jul 12 13:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:23:27.949479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a39be-146b-4d39-b2dd-0ea4b8b9a9ef None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:27.949775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a39be-146b-4d39-b2dd-0ea4b8b9a9ef None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:27.950059 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988a39be-146b-4d39-b2dd-0ea4b8b9a9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:27.979903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:28.024330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-988a39be-146b-4d39-b2dd-0ea4b8b9a9ef None None] MFA Rules not processed for user `b4f12d08d7084387a1bea7865bf77d0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:23:28.025522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-988a39be-146b-4d39-b2dd-0ea4b8b9a9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:28.104984 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 750/1504] 10.222.0.23 () {58 vars in 1060 bytes} [Tue Jul 12 13:23:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:23:28.112808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e48fb-cafc-4007-a340-2da1bd012b36 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:28.114981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-317e48fb-cafc-4007-a340-2da1bd012b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:28.164050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:28.218791 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e48fb-cafc-4007-a340-2da1bd012b36 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:28.292017 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:28.292606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:28.292890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:28.293167 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:28.294843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:28.392393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:28.393865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:28.397489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:28.412614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 755/1505] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:23:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:23:28.457651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0c94e-ed6c-48b3-b0a3-749a0769e620 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:28.457941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0c94e-ed6c-48b3-b0a3-749a0769e620 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:28.458214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d0c94e-ed6c-48b3-b0a3-749a0769e620 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:28.459481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 751/1506] 10.222.0.23 () {52 vars in 829 bytes} [Tue Jul 12 13:23:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:23:28.463108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:28.464459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:28.565350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c08877b-ff74-4168-a2d3-2a776229920b service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:28.657146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:28.658049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:28.658328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:28.658599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] PATH_INFO: `/v3/users/90bade92657d4302b2fcb8969112d13c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:28.671452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] RBAC: Authorizing `identity:delete_user(user_id=90bade92657d4302b2fcb8969112d13c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:28.672493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:23:28.672493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 12 13:23:28.672493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: Jul 12 13:23:28.673756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:28.683452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:28.737337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90bade92657d4302b2fcb8969112d13c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:28.749233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90bade92657d4302b2fcb8969112d13c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:28.749805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90bade92657d4302b2fcb8969112d13c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:28.767553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90bade92657d4302b2fcb8969112d13c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:28.777062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90bade92657d4302b2fcb8969112d13c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:28.782538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 756/1507] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:23:28 2022] DELETE /identity/v3/users/90bade92657d4302b2fcb8969112d13c => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:23:28.787843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18e9eb-5930-4097-aed0-3d86f0f4847e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:28.788476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18e9eb-5930-4097-aed0-3d86f0f4847e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:28.788476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba18e9eb-5930-4097-aed0-3d86f0f4847e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:28.843301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ba18e9eb-5930-4097-aed0-3d86f0f4847e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:23:28.844445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ba18e9eb-5930-4097-aed0-3d86f0f4847e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:28.920178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:28.932739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 752/1508] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:23:28.943311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:28.944593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:29.042562 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d920c35-6694-4d43-b97b-6cf17c16365a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:29.099216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:29.119729 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6597af9-8864-4c40-85b5-72dfec50152d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:29.120378 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6597af9-8864-4c40-85b5-72dfec50152d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:29.120658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6597af9-8864-4c40-85b5-72dfec50152d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:29.120932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6597af9-8864-4c40-85b5-72dfec50152d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:29.122600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e6597af9-8864-4c40-85b5-72dfec50152d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:29.227480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6597af9-8864-4c40-85b5-72dfec50152d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:29.228929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6597af9-8864-4c40-85b5-72dfec50152d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:29.247871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 757/1509] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:23:29.584518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18e9eb-5930-4097-aed0-3d86f0f4847e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:23:29.585835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ba18e9eb-5930-4097-aed0-3d86f0f4847e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:23:29.598013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba18e9eb-5930-4097-aed0-3d86f0f4847e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:23:29.696735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:23:29.697547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:23:29.697838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:23:29.698113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] PATH_INFO: `/v3/projects/5fc69c3343de413692f464989deb5b80` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:23:29.708800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fc69c3343de413692f464989deb5b80)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:23:29.709863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:23:29.709863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 12 13:23:29.709863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: Jul 12 13:23:29.711113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:23:29.767492 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fc69c3343de413692f464989deb5b80', 'request_id': 'req-ad76d21d-4909-47cc-9553-1b839a639921'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:29.781552 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fc69c3343de413692f464989deb5b80', 'request_id': 'req-ad76d21d-4909-47cc-9553-1b839a639921'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:29.790937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fc69c3343de413692f464989deb5b80', 'request_id': 'req-ad76d21d-4909-47cc-9553-1b839a639921'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:29.814413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] The token cache is being invalidate because project 5fc69c3343de413692f464989deb5b80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:23:29.814703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:29.815240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ad76d21d-4909-47cc-9553-1b839a639921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:23:29.817136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 753/1510] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:23:29 2022] DELETE /identity/v3/projects/5fc69c3343de413692f464989deb5b80 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:23:39.245354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=62 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:23:39.812015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:25:08.317587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36444b86-9251-4c97-9f73-f1b68ad84c76 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:08.318168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36444b86-9251-4c97-9f73-f1b68ad84c76 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:08.318168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36444b86-9251-4c97-9f73-f1b68ad84c76 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:08.319447 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:25:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:25:08.324359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd9834-d50c-4212-9d1f-7d6e3b912956 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:08.324811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd9834-d50c-4212-9d1f-7d6e3b912956 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:08.324953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fd9834-d50c-4212-9d1f-7d6e3b912956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:08.407944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b6fd9834-d50c-4212-9d1f-7d6e3b912956 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:08.409171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b6fd9834-d50c-4212-9d1f-7d6e3b912956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:08.512701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:25:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:25:08.665602 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36444b86-9251-4c97-9f73-f1b68ad84c76 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:25:08.667704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-36444b86-9251-4c97-9f73-f1b68ad84c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:08.784663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36444b86-9251-4c97-9f73-f1b68ad84c76 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:25:08.790806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893a3911-353a-4270-9b78-139566b65048 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:25:08.791462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3911-353a-4270-9b78-139566b65048 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:08.791753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3911-353a-4270-9b78-139566b65048 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:08.792037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893a3911-353a-4270-9b78-139566b65048 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:08.800133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a3911-353a-4270-9b78-139566b65048 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:25:08.802682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893a3911-353a-4270-9b78-139566b65048 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:25:08.812678 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 759/1513] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:25:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:25:09.636684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c608edf-a6bd-429a-b6c5-378675ef89c8 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:09.637216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c608edf-a6bd-429a-b6c5-378675ef89c8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:09.637216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c608edf-a6bd-429a-b6c5-378675ef89c8 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:09.638592 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:25:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:25:09.643414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588444-e713-4236-b15e-344c01518920 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:09.643871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588444-e713-4236-b15e-344c01518920 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:09.643981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02588444-e713-4236-b15e-344c01518920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:09.664439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-02588444-e713-4236-b15e-344c01518920 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:09.665675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-02588444-e713-4236-b15e-344c01518920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:09.673479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:25:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:25:09.823160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c608edf-a6bd-429a-b6c5-378675ef89c8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:25:09.838561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c608edf-a6bd-429a-b6c5-378675ef89c8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:25:09.843980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c38b5f-9b4d-4ac9-b34f-a8be727db157 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:25:09.844625 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c38b5f-9b4d-4ac9-b34f-a8be727db157 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:09.844908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c38b5f-9b4d-4ac9-b34f-a8be727db157 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:09.845188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c38b5f-9b4d-4ac9-b34f-a8be727db157 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:09.853992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c38b5f-9b4d-4ac9-b34f-a8be727db157 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:25:09.855754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c38b5f-9b4d-4ac9-b34f-a8be727db157 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:25:09.864864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 756/1516] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:25:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:25:10.317011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080d5dc-02ca-41ae-a764-31be7ef299c5 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:10.317300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080d5dc-02ca-41ae-a764-31be7ef299c5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:10.317606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2080d5dc-02ca-41ae-a764-31be7ef299c5 None None] PATH_INFO: `/v3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:10.318926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 761/1517] 10.222.0.25 () {54 vars in 925 bytes} [Tue Jul 12 13:25:10 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:25:10.325556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7439e1-d21e-4b0f-9cea-23d0f8f75b3e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:10.325847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7439e1-d21e-4b0f-9cea-23d0f8f75b3e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:10.326121 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7439e1-d21e-4b0f-9cea-23d0f8f75b3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:10.385517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4c7439e1-d21e-4b0f-9cea-23d0f8f75b3e None None] MFA Rules not processed for user `2e97119806774c2482a9886b11ab6f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:10.386733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4c7439e1-d21e-4b0f-9cea-23d0f8f75b3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:10.403213 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:25:10.457081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 757/1518] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Jul 12 13:25:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:25:19.314161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdaaca-3715-4aa1-8119-3c6ca60a785c None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:19.314625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdaaca-3715-4aa1-8119-3c6ca60a785c None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:19.314625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbdaaca-3715-4aa1-8119-3c6ca60a785c None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:19.316058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 762/1519] 10.222.0.23 () {54 vars in 960 bytes} [Tue Jul 12 13:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:25:19.321176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a885586-c391-4834-8f23-3e4bfcae07d2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:19.321583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a885586-c391-4834-8f23-3e4bfcae07d2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:19.321767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a885586-c391-4834-8f23-3e4bfcae07d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:19.341588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1a885586-c391-4834-8f23-3e4bfcae07d2 None None] MFA Rules not processed for user `2e97119806774c2482a9886b11ab6f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:19.342811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1a885586-c391-4834-8f23-3e4bfcae07d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:19.351805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 758/1520] 10.222.0.23 () {58 vars in 1057 bytes} [Tue Jul 12 13:25:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:25:19.359265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdaaca-3715-4aa1-8119-3c6ca60a785c None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:25:19.372906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbdaaca-3715-4aa1-8119-3c6ca60a785c None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:25:19.377752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4aa401-0e24-4296-b8a5-afa90ad65334 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e97119806774c2482a9886b11ab6f70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:25:19.378376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aa401-0e24-4296-b8a5-afa90ad65334 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:19.378705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aa401-0e24-4296-b8a5-afa90ad65334 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:19.378977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4aa401-0e24-4296-b8a5-afa90ad65334 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:19.387087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4aa401-0e24-4296-b8a5-afa90ad65334 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:25:19.388640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4aa401-0e24-4296-b8a5-afa90ad65334 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:25:19.398676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 763/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:25:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:25:21.105248 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0605017-a044-40f8-8c5f-0928a632d98a None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:21.105832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0605017-a044-40f8-8c5f-0928a632d98a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:21.105832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0605017-a044-40f8-8c5f-0928a632d98a None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:21.107220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:25:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:25:21.112034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c727c8-8cd1-46e6-912c-a4fa16aafc1d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:21.112424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c727c8-8cd1-46e6-912c-a4fa16aafc1d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:21.112593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c727c8-8cd1-46e6-912c-a4fa16aafc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:21.131473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-63c727c8-8cd1-46e6-912c-a4fa16aafc1d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:21.132795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-63c727c8-8cd1-46e6-912c-a4fa16aafc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:21.140879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:25:21.295070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0605017-a044-40f8-8c5f-0928a632d98a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:25:21.309843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0605017-a044-40f8-8c5f-0928a632d98a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:25:21.315464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840b644-cb7f-4304-894d-c488edf3fa46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:25:21.316134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840b644-cb7f-4304-894d-c488edf3fa46 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:21.316445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840b644-cb7f-4304-894d-c488edf3fa46 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:21.316724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840b644-cb7f-4304-894d-c488edf3fa46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:21.324670 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840b644-cb7f-4304-894d-c488edf3fa46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:25:21.326293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840b644-cb7f-4304-894d-c488edf3fa46 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:25:21.335622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 760/1524] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:25:22.152591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6c832-eaf5-4934-95f1-6bbac92a8e2f None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:22.153107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6c832-eaf5-4934-95f1-6bbac92a8e2f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:22.153107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6c832-eaf5-4934-95f1-6bbac92a8e2f None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:22.154567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Jul 12 13:25:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:25:22.159438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec63a4-d518-4f0a-97fb-aaaadb4e924c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:22.160444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec63a4-d518-4f0a-97fb-aaaadb4e924c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:22.160566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec63a4-d518-4f0a-97fb-aaaadb4e924c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:22.299162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-19ec63a4-d518-4f0a-97fb-aaaadb4e924c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:22.300930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-19ec63a4-d518-4f0a-97fb-aaaadb4e924c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:22.310697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Jul 12 13:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:25:22.463362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6c832-eaf5-4934-95f1-6bbac92a8e2f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:25:22.478283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6c832-eaf5-4934-95f1-6bbac92a8e2f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:25:22.483590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a2d52c-9cb6-4788-ba1c-a740e2fb20ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:25:22.484217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2d52c-9cb6-4788-ba1c-a740e2fb20ab service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:22.484499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2d52c-9cb6-4788-ba1c-a740e2fb20ab service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:22.484774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a2d52c-9cb6-4788-ba1c-a740e2fb20ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:22.493472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2d52c-9cb6-4788-ba1c-a740e2fb20ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:25:22.495041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a2d52c-9cb6-4788-ba1c-a740e2fb20ab service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:25:22.505125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 766/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:25:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:25:26.552433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1fc06-f0c5-4cc0-9c17-81fb8a0f14ac None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:26.552999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1fc06-f0c5-4cc0-9c17-81fb8a0f14ac None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:26.552999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda1fc06-f0c5-4cc0-9c17-81fb8a0f14ac None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:26.554308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:25:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:25:26.559699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bd9ac-2a9e-47dd-b0b1-2b901bfd3231 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:26.560123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bd9ac-2a9e-47dd-b0b1-2b901bfd3231 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:26.560258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36bd9ac-2a9e-47dd-b0b1-2b901bfd3231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:26.578572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a36bd9ac-2a9e-47dd-b0b1-2b901bfd3231 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:26.579785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a36bd9ac-2a9e-47dd-b0b1-2b901bfd3231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:26.589442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:25:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:25:26.663360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1fc06-f0c5-4cc0-9c17-81fb8a0f14ac None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:25:26.665069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fda1fc06-f0c5-4cc0-9c17-81fb8a0f14ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:26.742837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda1fc06-f0c5-4cc0-9c17-81fb8a0f14ac None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:25:26.746324 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ebe3c-5ef9-45ad-95b9-ba2bda2d26ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:25:26.746971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ebe3c-5ef9-45ad-95b9-ba2bda2d26ce service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:26.747254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ebe3c-5ef9-45ad-95b9-ba2bda2d26ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:26.747531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ebe3c-5ef9-45ad-95b9-ba2bda2d26ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:26.757364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ebe3c-5ef9-45ad-95b9-ba2bda2d26ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:25:26.758827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ebe3c-5ef9-45ad-95b9-ba2bda2d26ce service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:25:26.768140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 763/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:25:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:25:28.788474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd794c20-4a47-434e-8531-13baf68179a9 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:28.789057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd794c20-4a47-434e-8531-13baf68179a9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:28.789057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd794c20-4a47-434e-8531-13baf68179a9 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:28.790421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:25:28.795619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaf78c-0a88-4038-8a1a-35e67d5becb9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:28.796035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaf78c-0a88-4038-8a1a-35e67d5becb9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:28.796170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebaf78c-0a88-4038-8a1a-35e67d5becb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:28.814302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-aebaf78c-0a88-4038-8a1a-35e67d5becb9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:28.815518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-aebaf78c-0a88-4038-8a1a-35e67d5becb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:28.823925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:25:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:25:28.859635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd794c20-4a47-434e-8531-13baf68179a9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:25:28.870440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd794c20-4a47-434e-8531-13baf68179a9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:25:28.873994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec68ff0b-b75f-4e9f-b89b-885e949fe012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:25:28.874630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68ff0b-b75f-4e9f-b89b-885e949fe012 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:28.874912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68ff0b-b75f-4e9f-b89b-885e949fe012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:28.875185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec68ff0b-b75f-4e9f-b89b-885e949fe012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:28.884450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68ff0b-b75f-4e9f-b89b-885e949fe012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:25:28.885910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec68ff0b-b75f-4e9f-b89b-885e949fe012 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:25:28.895571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 769/1533] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:25:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:25:32.452845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07cee8-aba1-46e1-ba7e-54c4b16f03fc None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:32.453426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07cee8-aba1-46e1-ba7e-54c4b16f03fc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:32.453426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c07cee8-aba1-46e1-ba7e-54c4b16f03fc None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:32.454764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:25:32.460167 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d14ea-23b6-431c-932d-81bd82b1667f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:32.460580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d14ea-23b6-431c-932d-81bd82b1667f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:32.460722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d14ea-23b6-431c-932d-81bd82b1667f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:32.466897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=81 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:25:32.479682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-071d14ea-23b6-431c-932d-81bd82b1667f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:32.480875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-071d14ea-23b6-431c-932d-81bd82b1667f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:32.489245 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:25:32.792255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07cee8-aba1-46e1-ba7e-54c4b16f03fc None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:25:32.793992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3c07cee8-aba1-46e1-ba7e-54c4b16f03fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:32.860237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:25:32.874682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c07cee8-aba1-46e1-ba7e-54c4b16f03fc None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:25:32.878495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b7270c-0d9f-4894-bd95-465e0602c6d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:25:32.879426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7270c-0d9f-4894-bd95-465e0602c6d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:32.879722 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7270c-0d9f-4894-bd95-465e0602c6d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:32.880000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b7270c-0d9f-4894-bd95-465e0602c6d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:32.888907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7270c-0d9f-4894-bd95-465e0602c6d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:25:32.890558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b7270c-0d9f-4894-bd95-465e0602c6d3 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:25:32.902416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 766/1536] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:25:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:25:34.981341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad3734-8f3b-4bad-9c89-4374f135f749 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:34.981970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad3734-8f3b-4bad-9c89-4374f135f749 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:34.981970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bad3734-8f3b-4bad-9c89-4374f135f749 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:34.983303 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Jul 12 13:25:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:25:34.989167 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e54939-902c-472d-a8b3-2486fee8cfad None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:34.989653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e54939-902c-472d-a8b3-2486fee8cfad None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:34.989761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e54939-902c-472d-a8b3-2486fee8cfad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:35.007091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-32e54939-902c-472d-a8b3-2486fee8cfad None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:25:35.008288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-32e54939-902c-472d-a8b3-2486fee8cfad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:35.016104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Jul 12 13:25:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:25:35.322208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad3734-8f3b-4bad-9c89-4374f135f749 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:25:35.324026 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7bad3734-8f3b-4bad-9c89-4374f135f749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:25:35.337133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bad3734-8f3b-4bad-9c89-4374f135f749 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:25:35.340561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c620e1e4-04c9-4cee-9d67-e19371755f55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:25:35.341196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620e1e4-04c9-4cee-9d67-e19371755f55 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:25:35.341518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620e1e4-04c9-4cee-9d67-e19371755f55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:25:35.341796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c620e1e4-04c9-4cee-9d67-e19371755f55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:25:35.350509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620e1e4-04c9-4cee-9d67-e19371755f55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:25:35.351917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c620e1e4-04c9-4cee-9d67-e19371755f55 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:25:35.361091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 772/1539] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:25:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:25:45.002251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=11 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:25:45.357081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=8 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:07.455349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579e92b-24fb-4b86-9468-b7c14cb54531 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.465278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579e92b-24fb-4b86-9468-b7c14cb54531 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.465278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c579e92b-24fb-4b86-9468-b7c14cb54531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.483869 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c579e92b-24fb-4b86-9468-b7c14cb54531 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:07.485204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c579e92b-24fb-4b86-9468-b7c14cb54531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:07.504548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79785d47-35c0-408e-abb5-e3992120160e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.505118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79785d47-35c0-408e-abb5-e3992120160e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.505118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79785d47-35c0-408e-abb5-e3992120160e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.507366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 768/1540] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:07.522925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff018bc-9ab0-4a2e-b616-96d84545f0a7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.523384 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff018bc-9ab0-4a2e-b616-96d84545f0a7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.523646 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff018bc-9ab0-4a2e-b616-96d84545f0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.536022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-79785d47-35c0-408e-abb5-e3992120160e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:07.542863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-79785d47-35c0-408e-abb5-e3992120160e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:07.551393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6ff018bc-9ab0-4a2e-b616-96d84545f0a7 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:07.552627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6ff018bc-9ab0-4a2e-b616-96d84545f0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:07.556771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 773/1541] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:27:07.567056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 769/1542] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:07.567281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7026a-2144-4a39-b9aa-f4d4dd03d284 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.567395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7026a-2144-4a39-b9aa-f4d4dd03d284 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.567653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e7026a-2144-4a39-b9aa-f4d4dd03d284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.573820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff018bc-9ab0-4a2e-b616-96d84545f0a7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:07.588566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff018bc-9ab0-4a2e-b616-96d84545f0a7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:07.594022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec4cca-c065-4ea9-88f4-3e535eef7b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:07.594933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec4cca-c065-4ea9-88f4-3e535eef7b64 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.595223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec4cca-c065-4ea9-88f4-3e535eef7b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.595500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec4cca-c065-4ea9-88f4-3e535eef7b64 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.596339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec4cca-c065-4ea9-88f4-3e535eef7b64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:07.596835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec4cca-c065-4ea9-88f4-3e535eef7b64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:07.603041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec4cca-c065-4ea9-88f4-3e535eef7b64 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:07.606417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b4e7026a-2144-4a39-b9aa-f4d4dd03d284 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:07.608025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b4e7026a-2144-4a39-b9aa-f4d4dd03d284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:07.621974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 770/1543] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:27:07.631741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec4cca-c065-4ea9-88f4-3e535eef7b64 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:07.638833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 774/1544] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:07.644802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7026a-2144-4a39-b9aa-f4d4dd03d284 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:07.656286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec4cca-c065-4ea9-88f4-3e535eef7b64 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:07.661250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bbf22-abe9-401f-8dc7-660c7797028d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:07.671883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bbf22-abe9-401f-8dc7-660c7797028d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.672308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bbf22-abe9-401f-8dc7-660c7797028d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.672600 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bbf22-abe9-401f-8dc7-660c7797028d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.673510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bbf22-abe9-401f-8dc7-660c7797028d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:07.674701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-ab6bbf22-abe9-401f-8dc7-660c7797028d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:27:07.674701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 12 13:27:07.674701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: Jul 12 13:27:07.675182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e7026a-2144-4a39-b9aa-f4d4dd03d284 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:07.676397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6bbf22-abe9-401f-8dc7-660c7797028d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:07.694133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:07.694822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.695108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.695384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.696123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:07.696438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:07.697490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:27:07.697490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 12 13:27:07.697490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: Jul 12 13:27:07.698393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:07.728896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 775/1545] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:07.738308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:07.751798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 771/1546] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:07 2022] POST /identity/v3/projects => generated 389 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:27:07.759603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bbf22-abe9-401f-8dc7-660c7797028d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:07.764547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91add795-0e96-40bf-b82b-255544f8a347 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:07.894632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bbf22-abe9-401f-8dc7-660c7797028d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:07.910506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002be5f-6cfc-47db-ab78-894bf1579e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:07.911225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002be5f-6cfc-47db-ab78-894bf1579e7b admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.911514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002be5f-6cfc-47db-ab78-894bf1579e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.911795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002be5f-6cfc-47db-ab78-894bf1579e7b admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.912507 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002be5f-6cfc-47db-ab78-894bf1579e7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:07.914628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5002be5f-6cfc-47db-ab78-894bf1579e7b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:07.944082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224575f-c8af-4fe2-a093-f6db74147490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:07.946029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224575f-c8af-4fe2-a093-f6db74147490 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.946415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224575f-c8af-4fe2-a093-f6db74147490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.946713 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224575f-c8af-4fe2-a093-f6db74147490 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.948112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 772/1547] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:07 2022] POST /identity/v3/users => generated 295 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:27:07.949045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224575f-c8af-4fe2-a093-f6db74147490 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:07.951134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabfe4c-f24b-4d1a-8622-9b91948e60e4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:07.952597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabfe4c-f24b-4d1a-8622-9b91948e60e4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:07.956732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabfe4c-f24b-4d1a-8622-9b91948e60e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:07.958381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224575f-c8af-4fe2-a093-f6db74147490 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.005648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1aabfe4c-f24b-4d1a-8622-9b91948e60e4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.006948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1aabfe4c-f24b-4d1a-8622-9b91948e60e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.019990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 776/1548] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:07 2022] POST /identity/v3/projects => generated 375 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:08.021260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c61b9-d4dd-4d5c-8ffa-3b80aecae535 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.021778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c61b9-d4dd-4d5c-8ffa-3b80aecae535 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.022698 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57c61b9-d4dd-4d5c-8ffa-3b80aecae535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.030665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 773/1549] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.031527 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabfe4c-f24b-4d1a-8622-9b91948e60e4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.052044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabfe4c-f24b-4d1a-8622-9b91948e60e4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.056120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d57c61b9-d4dd-4d5c-8ffa-3b80aecae535 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.057539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d57c61b9-d4dd-4d5c-8ffa-3b80aecae535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.130943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25fe9c7-41bf-4554-af30-702d0f23e16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.131655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fe9c7-41bf-4554-af30-702d0f23e16c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.131920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fe9c7-41bf-4554-af30-702d0f23e16c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.132198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25fe9c7-41bf-4554-af30-702d0f23e16c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.132952 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fe9c7-41bf-4554-af30-702d0f23e16c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.135259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25fe9c7-41bf-4554-af30-702d0f23e16c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.146068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 774/1550] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:08.149632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b247c1-cd98-4c1d-a8e0-537e695196e1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.149740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 777/1551] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.149845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b247c1-cd98-4c1d-a8e0-537e695196e1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.149845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b247c1-cd98-4c1d-a8e0-537e695196e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.150021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694164ba-d09c-4756-8da1-54375be95acb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.150021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694164ba-d09c-4756-8da1-54375be95acb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.150021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694164ba-d09c-4756-8da1-54375be95acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.169018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f5b247c1-cd98-4c1d-a8e0-537e695196e1 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.170290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f5b247c1-cd98-4c1d-a8e0-537e695196e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.173588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-694164ba-d09c-4756-8da1-54375be95acb None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.176156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-694164ba-d09c-4756-8da1-54375be95acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.179409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 775/1552] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.180207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b247c1-cd98-4c1d-a8e0-537e695196e1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.189668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 778/1553] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.191102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b247c1-cd98-4c1d-a8e0-537e695196e1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.191475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0c3d1-dbfe-44a2-afcb-0ae24e51387d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.191844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0c3d1-dbfe-44a2-afcb-0ae24e51387d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.192185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f0c3d1-dbfe-44a2-afcb-0ae24e51387d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.206341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b397e0-0c3b-47a4-9e5e-61f96849f600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.207024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b397e0-0c3b-47a4-9e5e-61f96849f600 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.207311 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b397e0-0c3b-47a4-9e5e-61f96849f600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.207590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b397e0-0c3b-47a4-9e5e-61f96849f600 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.208289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b397e0-0c3b-47a4-9e5e-61f96849f600 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.210280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b397e0-0c3b-47a4-9e5e-61f96849f600 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.220482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-00f0c3d1-dbfe-44a2-afcb-0ae24e51387d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.222272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-00f0c3d1-dbfe-44a2-afcb-0ae24e51387d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.229183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:08.235534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 779/1554] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.240312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 776/1555] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/users => generated 288 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:08.240525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db6168-bc40-487c-a1a4-7a6923f2918a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.240525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db6168-bc40-487c-a1a4-7a6923f2918a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.240717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d387541-94f7-4edd-b385-ee6ef748bdc5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.240829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db6168-bc40-487c-a1a4-7a6923f2918a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.240924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d387541-94f7-4edd-b385-ee6ef748bdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.240924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d387541-94f7-4edd-b385-ee6ef748bdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.263247 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5d387541-94f7-4edd-b385-ee6ef748bdc5 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.263647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-93db6168-bc40-487c-a1a4-7a6923f2918a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.264671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5d387541-94f7-4edd-b385-ee6ef748bdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.265633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-93db6168-bc40-487c-a1a4-7a6923f2918a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.273934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 777/1556] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.274666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d387541-94f7-4edd-b385-ee6ef748bdc5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.278134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 780/1557] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.278134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7da77-10d4-4a4b-b49e-9e5102754dca None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.278134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7da77-10d4-4a4b-b49e-9e5102754dca None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.278134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f7da77-10d4-4a4b-b49e-9e5102754dca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.290321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d387541-94f7-4edd-b385-ee6ef748bdc5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.295037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20f127-1bb9-4686-a0f5-4ddc81ed8746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.295704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20f127-1bb9-4686-a0f5-4ddc81ed8746 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.295986 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20f127-1bb9-4686-a0f5-4ddc81ed8746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.296261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20f127-1bb9-4686-a0f5-4ddc81ed8746 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.297331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20f127-1bb9-4686-a0f5-4ddc81ed8746 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.298847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20f127-1bb9-4686-a0f5-4ddc81ed8746 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.300934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-27f7da77-10d4-4a4b-b49e-9e5102754dca None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.303168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-27f7da77-10d4-4a4b-b49e-9e5102754dca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.310417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 778/1558] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:08.311946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34ae79-9c8d-4d52-be7f-bd9d16ff2e46 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.312543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34ae79-9c8d-4d52-be7f-bd9d16ff2e46 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.312883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e34ae79-9c8d-4d52-be7f-bd9d16ff2e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.317857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 781/1559] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.319414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f583609-f1e7-4afb-ae0c-83dd4c392bdd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.319710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f583609-f1e7-4afb-ae0c-83dd4c392bdd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.319994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f583609-f1e7-4afb-ae0c-83dd4c392bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.339681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4f583609-f1e7-4afb-ae0c-83dd4c392bdd None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.342000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4f583609-f1e7-4afb-ae0c-83dd4c392bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.345288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-2e34ae79-9c8d-4d52-be7f-bd9d16ff2e46 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.346577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2e34ae79-9c8d-4d52-be7f-bd9d16ff2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.350685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 782/1560] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.354406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6b855-6b39-4b80-8bfa-eea92e53f48f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.354406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6b855-6b39-4b80-8bfa-eea92e53f48f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.354406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6b855-6b39-4b80-8bfa-eea92e53f48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.355667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 779/1561] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.356559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34ae79-9c8d-4d52-be7f-bd9d16ff2e46 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.369603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e34ae79-9c8d-4d52-be7f-bd9d16ff2e46 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.372191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e1d6b855-6b39-4b80-8bfa-eea92e53f48f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.373416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e1d6b855-6b39-4b80-8bfa-eea92e53f48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.374370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bacb0-9a7e-4802-b09e-9b1a73a6e025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.375044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bacb0-9a7e-4802-b09e-9b1a73a6e025 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.375328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bacb0-9a7e-4802-b09e-9b1a73a6e025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.375611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bacb0-9a7e-4802-b09e-9b1a73a6e025 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.376344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bacb0-9a7e-4802-b09e-9b1a73a6e025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.378616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bacb0-9a7e-4802-b09e-9b1a73a6e025 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.383316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 783/1562] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.384213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6b855-6b39-4b80-8bfa-eea92e53f48f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.390285 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 780/1563] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:08.391870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0882e2-6821-4df7-bd79-3e81d328a7c2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.392236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0882e2-6821-4df7-bd79-3e81d328a7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.392585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0882e2-6821-4df7-bd79-3e81d328a7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.396460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d6b855-6b39-4b80-8bfa-eea92e53f48f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.401203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed4264-fc4e-4296-a408-bc5979b280b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.401929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed4264-fc4e-4296-a408-bc5979b280b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.402217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed4264-fc4e-4296-a408-bc5979b280b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.402499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ed4264-fc4e-4296-a408-bc5979b280b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.403242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed4264-fc4e-4296-a408-bc5979b280b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:08.403553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed4264-fc4e-4296-a408-bc5979b280b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.405240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ed4264-fc4e-4296-a408-bc5979b280b3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.414444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ae0882e2-6821-4df7-bd79-3e81d328a7c2 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.415671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ae0882e2-6821-4df7-bd79-3e81d328a7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.423728 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 781/1564] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.433456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 784/1565] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:08.433765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0882e2-6821-4df7-bd79-3e81d328a7c2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.433933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed4264-fc4e-4296-a408-bc5979b280b3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.442159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0882e2-6821-4df7-bd79-3e81d328a7c2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.443254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ed4264-fc4e-4296-a408-bc5979b280b3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.447304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3740b-c54c-419e-b929-b94d67a1737f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.448234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3740b-c54c-419e-b929-b94d67a1737f admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.448520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3740b-c54c-419e-b929-b94d67a1737f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.448798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3740b-c54c-419e-b929-b94d67a1737f admin admin] PATH_INFO: `/v3/projects/0a2414687e6d4a449e3aeec65921b3c4/users/f8de4128718d4837af77d427973d30be/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.450725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaa7e1-3bf4-4018-950e-b2b3199403aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.451811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaa7e1-3bf4-4018-950e-b2b3199403aa admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.452245 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaa7e1-3bf4-4018-950e-b2b3199403aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.452643 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbaa7e1-3bf4-4018-950e-b2b3199403aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.460813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbaa7e1-3bf4-4018-950e-b2b3199403aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:08.460813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbaa7e1-3bf4-4018-950e-b2b3199403aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.460813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbaa7e1-3bf4-4018-950e-b2b3199403aa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.460813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:08.466694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3740b-c54c-419e-b929-b94d67a1737f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a2414687e6d4a449e3aeec65921b3c4, user_id=f8de4128718d4837af77d427973d30be, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.467744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-68f3740b-c54c-419e-b929-b94d67a1737f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:27:08.467744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 12 13:27:08.467744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: Jul 12 13:27:08.469716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3740b-c54c-419e-b929-b94d67a1737f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.480708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 785/1566] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:08.482190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaa7e1-3bf4-4018-950e-b2b3199403aa admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.486813 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 782/1567] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:08 2022] PUT /identity/v3/projects/0a2414687e6d4a449e3aeec65921b3c4/users/f8de4128718d4837af77d427973d30be/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:08.487700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3740b-c54c-419e-b929-b94d67a1737f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.496873 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbaa7e1-3bf4-4018-950e-b2b3199403aa admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.559585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3740b-c54c-419e-b929-b94d67a1737f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.564534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45a6da-f4f2-42df-92c6-e905f9bd7e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.565318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45a6da-f4f2-42df-92c6-e905f9bd7e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.565689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45a6da-f4f2-42df-92c6-e905f9bd7e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.565965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a45a6da-f4f2-42df-92c6-e905f9bd7e0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.566745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45a6da-f4f2-42df-92c6-e905f9bd7e0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:08.567012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45a6da-f4f2-42df-92c6-e905f9bd7e0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.567366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ac01d-6308-4f71-aac6-fe53141dfbb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.568044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ac01d-6308-4f71-aac6-fe53141dfbb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.568338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ac01d-6308-4f71-aac6-fe53141dfbb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.568617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81ac01d-6308-4f71-aac6-fe53141dfbb4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.568731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a45a6da-f4f2-42df-92c6-e905f9bd7e0c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.569403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ac01d-6308-4f71-aac6-fe53141dfbb4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:08.569736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ac01d-6308-4f71-aac6-fe53141dfbb4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.571418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81ac01d-6308-4f71-aac6-fe53141dfbb4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.585550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 783/1568] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:08.586404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45a6da-f4f2-42df-92c6-e905f9bd7e0c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.586541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 786/1569] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:08.587305 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ac01d-6308-4f71-aac6-fe53141dfbb4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.598807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a45a6da-f4f2-42df-92c6-e905f9bd7e0c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.605551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9492f-96da-4bb2-b498-949ee651ec96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.606235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9492f-96da-4bb2-b498-949ee651ec96 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.606365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81ac01d-6308-4f71-aac6-fe53141dfbb4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.606519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9492f-96da-4bb2-b498-949ee651ec96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.606794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9492f-96da-4bb2-b498-949ee651ec96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.607540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9492f-96da-4bb2-b498-949ee651ec96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:08.607844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9492f-96da-4bb2-b498-949ee651ec96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.609640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9492f-96da-4bb2-b498-949ee651ec96 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.611480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83427f48-8270-4501-82c1-a14f9324278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.612216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83427f48-8270-4501-82c1-a14f9324278b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.612901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83427f48-8270-4501-82c1-a14f9324278b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.612901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83427f48-8270-4501-82c1-a14f9324278b admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.614226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83427f48-8270-4501-82c1-a14f9324278b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.616639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83427f48-8270-4501-82c1-a14f9324278b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.624558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 784/1570] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:08.625613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9492f-96da-4bb2-b498-949ee651ec96 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.630011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 787/1571] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:08.630766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83427f48-8270-4501-82c1-a14f9324278b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.637955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9492f-96da-4bb2-b498-949ee651ec96 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.642108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83427f48-8270-4501-82c1-a14f9324278b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.645120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50285ed0-07b4-450a-8c88-ac77a879b9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.645862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50285ed0-07b4-450a-8c88-ac77a879b9e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.646150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50285ed0-07b4-450a-8c88-ac77a879b9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.646427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50285ed0-07b4-450a-8c88-ac77a879b9e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.647326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50285ed0-07b4-450a-8c88-ac77a879b9e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:08.647776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50285ed0-07b4-450a-8c88-ac77a879b9e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.648947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65b3b9-80a9-421c-80be-f6612184d980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.649642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65b3b9-80a9-421c-80be-f6612184d980 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.649769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50285ed0-07b4-450a-8c88-ac77a879b9e3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.649927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65b3b9-80a9-421c-80be-f6612184d980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.650204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65b3b9-80a9-421c-80be-f6612184d980 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.651294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65b3b9-80a9-421c-80be-f6612184d980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.653248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65b3b9-80a9-421c-80be-f6612184d980 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.663607 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 785/1572] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:08.664444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50285ed0-07b4-450a-8c88-ac77a879b9e3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.675025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50285ed0-07b4-450a-8c88-ac77a879b9e3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.680006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68e816-09de-42e5-a3fa-529f29eed263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.680697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68e816-09de-42e5-a3fa-529f29eed263 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.680933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68e816-09de-42e5-a3fa-529f29eed263 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.681409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d68e816-09de-42e5-a3fa-529f29eed263 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.682124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68e816-09de-42e5-a3fa-529f29eed263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.684040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d68e816-09de-42e5-a3fa-529f29eed263 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.694284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 788/1573] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/projects => generated 403 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:08.694284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7ba18-cff6-45fe-8aab-3fc5f8b9d0bd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.694284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7ba18-cff6-45fe-8aab-3fc5f8b9d0bd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.694284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7ba18-cff6-45fe-8aab-3fc5f8b9d0bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.712282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9eb7ba18-cff6-45fe-8aab-3fc5f8b9d0bd None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:08.713591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9eb7ba18-cff6-45fe-8aab-3fc5f8b9d0bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:08.717810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 786/1574] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:08.719337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68e816-09de-42e5-a3fa-529f29eed263 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.773526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:08.800392 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 789/1575] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:08.801334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7ba18-cff6-45fe-8aab-3fc5f8b9d0bd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.810676 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d68e816-09de-42e5-a3fa-529f29eed263 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.810861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb7ba18-cff6-45fe-8aab-3fc5f8b9d0bd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.816092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306534b-9add-4b87-83cb-abe01b006a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.816766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306534b-9add-4b87-83cb-abe01b006a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.817004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa00ea3-497b-4c84-ae0b-10c1f15a4fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.817124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306534b-9add-4b87-83cb-abe01b006a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.817319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306534b-9add-4b87-83cb-abe01b006a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.817736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa00ea3-497b-4c84-ae0b-10c1f15a4fc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.818042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa00ea3-497b-4c84-ae0b-10c1f15a4fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.818146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306534b-9add-4b87-83cb-abe01b006a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.818322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa00ea3-497b-4c84-ae0b-10c1f15a4fc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.819019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa00ea3-497b-4c84-ae0b-10c1f15a4fc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.819958 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306534b-9add-4b87-83cb-abe01b006a1f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.820929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa00ea3-497b-4c84-ae0b-10c1f15a4fc4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.854290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 787/1576] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/projects => generated 363 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:08.855315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa00ea3-497b-4c84-ae0b-10c1f15a4fc4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.857207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 790/1577] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:08.858481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306534b-9add-4b87-83cb-abe01b006a1f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.909308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:08.949641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa00ea3-497b-4c84-ae0b-10c1f15a4fc4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.951038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306534b-9add-4b87-83cb-abe01b006a1f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:08.955235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a521c53-956f-4073-8e8e-79b3efc9cdbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.956449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a521c53-956f-4073-8e8e-79b3efc9cdbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.956737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a521c53-956f-4073-8e8e-79b3efc9cdbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.957193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a521c53-956f-4073-8e8e-79b3efc9cdbb admin admin] PATH_INFO: `/v3/projects/ab8fca4ab6ed466a864c5551fea5f7ff/users/c4b710bbdf0c4d58b4f3ee7ef7410c0a/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.958275 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ac9c-0ca8-40f9-a915-24c91b739561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:08.958960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ac9c-0ca8-40f9-a915-24c91b739561 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:08.959336 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ac9c-0ca8-40f9-a915-24c91b739561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:08.959539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd68ac9c-0ca8-40f9-a915-24c91b739561 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:08.960236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68ac9c-0ca8-40f9-a915-24c91b739561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.962314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd68ac9c-0ca8-40f9-a915-24c91b739561 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.974225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a521c53-956f-4073-8e8e-79b3efc9cdbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab8fca4ab6ed466a864c5551fea5f7ff, user_id=c4b710bbdf0c4d58b4f3ee7ef7410c0a, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:08.977478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a521c53-956f-4073-8e8e-79b3efc9cdbb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:08.995613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 788/1578] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:08 2022] PUT /identity/v3/projects/ab8fca4ab6ed466a864c5551fea5f7ff/users/c4b710bbdf0c4d58b4f3ee7ef7410c0a/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:08.999590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 791/1579] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/projects => generated 375 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:08.999741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a521c53-956f-4073-8e8e-79b3efc9cdbb admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:08.999839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ac9c-0ca8-40f9-a915-24c91b739561 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.071465 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd68ac9c-0ca8-40f9-a915-24c91b739561 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.074803 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a521c53-956f-4073-8e8e-79b3efc9cdbb admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.076780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfa1da-29d3-434d-9351-bbffdc420979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.077529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfa1da-29d3-434d-9351-bbffdc420979 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.077850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfa1da-29d3-434d-9351-bbffdc420979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.078164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bfa1da-29d3-434d-9351-bbffdc420979 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.078965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfa1da-29d3-434d-9351-bbffdc420979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.080319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c362372-eb8a-4687-88ae-115461b40d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.081953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-e2bfa1da-29d3-434d-9351-bbffdc420979 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:27:09.081953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 12 13:27:09.081953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: Jul 12 13:27:09.082705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c362372-eb8a-4687-88ae-115461b40d2a admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.082705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c362372-eb8a-4687-88ae-115461b40d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.083434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bfa1da-29d3-434d-9351-bbffdc420979 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.083549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c362372-eb8a-4687-88ae-115461b40d2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.083549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c362372-eb8a-4687-88ae-115461b40d2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.084833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c362372-eb8a-4687-88ae-115461b40d2a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.107748 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 792/1580] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/users => generated 302 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:09.107748 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfa1da-29d3-434d-9351-bbffdc420979 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.119028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bfa1da-29d3-434d-9351-bbffdc420979 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.123229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 789/1581] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:08 2022] POST /identity/v3/projects => generated 391 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:09.124143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c362372-eb8a-4687-88ae-115461b40d2a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.126934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e3440-a188-462b-a32d-32b3b2492e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.127679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e3440-a188-462b-a32d-32b3b2492e7e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.128179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e3440-a188-462b-a32d-32b3b2492e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.128657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9e3440-a188-462b-a32d-32b3b2492e7e admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.129361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e3440-a188-462b-a32d-32b3b2492e7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.131447 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9e3440-a188-462b-a32d-32b3b2492e7e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.155056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 793/1582] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:09 2022] POST /identity/v3/users => generated 291 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:09.155056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e3440-a188-462b-a32d-32b3b2492e7e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.164683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:09.165728 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9e3440-a188-462b-a32d-32b3b2492e7e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.191772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c362372-eb8a-4687-88ae-115461b40d2a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.195648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f9e95f-ed82-45bb-9795-22e90654bed8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.196355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9e95f-ed82-45bb-9795-22e90654bed8 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.196643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9e95f-ed82-45bb-9795-22e90654bed8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.196946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f9e95f-ed82-45bb-9795-22e90654bed8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.236316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0abf77-09e6-41bf-bc91-6476d5655a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.237658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0abf77-09e6-41bf-bc91-6476d5655a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.237949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0abf77-09e6-41bf-bc91-6476d5655a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.238229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0abf77-09e6-41bf-bc91-6476d5655a75 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.238938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0abf77-09e6-41bf-bc91-6476d5655a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.240966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0abf77-09e6-41bf-bc91-6476d5655a75 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.266161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 794/1583] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:09 2022] POST /identity/v3/users => generated 282 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:09.266161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0abf77-09e6-41bf-bc91-6476d5655a75 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.276098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0abf77-09e6-41bf-bc91-6476d5655a75 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.282312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9bb90-34f6-4365-bf92-cb74d30038fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.283033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9bb90-34f6-4365-bf92-cb74d30038fc admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.283312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9bb90-34f6-4365-bf92-cb74d30038fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.284136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f9bb90-34f6-4365-bf92-cb74d30038fc admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.284719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9e95f-ed82-45bb-9795-22e90654bed8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.284834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9bb90-34f6-4365-bf92-cb74d30038fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.286239 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f9e95f-ed82-45bb-9795-22e90654bed8 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.286934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f9bb90-34f6-4365-bf92-cb74d30038fc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.296347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 790/1584] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:09.298581 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9581a2-2bc9-47f0-aa2b-6ec7bccd3f3c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.298893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9581a2-2bc9-47f0-aa2b-6ec7bccd3f3c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.299821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9581a2-2bc9-47f0-aa2b-6ec7bccd3f3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.322022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 795/1585] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:09 2022] POST /identity/v3/users => generated 290 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:09.322022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9bb90-34f6-4365-bf92-cb74d30038fc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.323920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:09.324641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9f9581a2-2bc9-47f0-aa2b-6ec7bccd3f3c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:09.325842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9f9581a2-2bc9-47f0-aa2b-6ec7bccd3f3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:09.334394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f9bb90-34f6-4365-bf92-cb74d30038fc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.336817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 791/1586] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:09.338933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9581a2-2bc9-47f0-aa2b-6ec7bccd3f3c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.344619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5f1a7-20df-4c28-a2b5-dba1448a1293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.353335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5f1a7-20df-4c28-a2b5-dba1448a1293 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.353601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9581a2-2bc9-47f0-aa2b-6ec7bccd3f3c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.353914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5f1a7-20df-4c28-a2b5-dba1448a1293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.354290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e5f1a7-20df-4c28-a2b5-dba1448a1293 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.355149 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5f1a7-20df-4c28-a2b5-dba1448a1293 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.357203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e5f1a7-20df-4c28-a2b5-dba1448a1293 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.358732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620a834-7487-4280-9011-daa8e3c488e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.359483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620a834-7487-4280-9011-daa8e3c488e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.360061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620a834-7487-4280-9011-daa8e3c488e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.360389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d620a834-7487-4280-9011-daa8e3c488e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.361260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620a834-7487-4280-9011-daa8e3c488e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.364092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d620a834-7487-4280-9011-daa8e3c488e7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.377673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 792/1587] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.378563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620a834-7487-4280-9011-daa8e3c488e7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.388961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d620a834-7487-4280-9011-daa8e3c488e7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.397232 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf27b9-968e-4c75-a6b8-07df2de62347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.397999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf27b9-968e-4c75-a6b8-07df2de62347 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.398741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf27b9-968e-4c75-a6b8-07df2de62347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.398741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf27b9-968e-4c75-a6b8-07df2de62347 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.399039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 796/1588] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:09 2022] POST /identity/v3/users => generated 288 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:09.399316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf27b9-968e-4c75-a6b8-07df2de62347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.400014 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5f1a7-20df-4c28-a2b5-dba1448a1293 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.401526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf27b9-968e-4c75-a6b8-07df2de62347 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.412709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e5f1a7-20df-4c28-a2b5-dba1448a1293 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.418480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34276fe9-d67c-4368-8ff8-1ef09bc52d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.419314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34276fe9-d67c-4368-8ff8-1ef09bc52d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.419700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34276fe9-d67c-4368-8ff8-1ef09bc52d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.420075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34276fe9-d67c-4368-8ff8-1ef09bc52d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.420934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34276fe9-d67c-4368-8ff8-1ef09bc52d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.422703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34276fe9-d67c-4368-8ff8-1ef09bc52d7a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.425494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 793/1589] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:09 2022] POST /identity/v3/users => generated 296 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:09.426349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf27b9-968e-4c75-a6b8-07df2de62347 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.442263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 797/1590] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.442263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34276fe9-d67c-4368-8ff8-1ef09bc52d7a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.442608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf27b9-968e-4c75-a6b8-07df2de62347 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.447406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f0cb7-a952-4d3a-afd9-6992012ca6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.447956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f0cb7-a952-4d3a-afd9-6992012ca6ed admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.448248 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f0cb7-a952-4d3a-afd9-6992012ca6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.449097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865f0cb7-a952-4d3a-afd9-6992012ca6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.450603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f0cb7-a952-4d3a-afd9-6992012ca6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.452305 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865f0cb7-a952-4d3a-afd9-6992012ca6ed admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.453055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34276fe9-d67c-4368-8ff8-1ef09bc52d7a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.458727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66697b24-fc35-4fb6-aa33-2ff2ac2086e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.459431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66697b24-fc35-4fb6-aa33-2ff2ac2086e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.459857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66697b24-fc35-4fb6-aa33-2ff2ac2086e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.460178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66697b24-fc35-4fb6-aa33-2ff2ac2086e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.461197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66697b24-fc35-4fb6-aa33-2ff2ac2086e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.462988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66697b24-fc35-4fb6-aa33-2ff2ac2086e9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.466896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 794/1591] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.467929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f0cb7-a952-4d3a-afd9-6992012ca6ed admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.476053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 798/1592] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.477073 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66697b24-fc35-4fb6-aa33-2ff2ac2086e9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.480835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-66697b24-fc35-4fb6-aa33-2ff2ac2086e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:09.481220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865f0cb7-a952-4d3a-afd9-6992012ca6ed admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.487700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307b6dc-ca90-4dcf-ae73-050ff650abef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.488475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307b6dc-ca90-4dcf-ae73-050ff650abef admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.488836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307b6dc-ca90-4dcf-ae73-050ff650abef admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.489200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2307b6dc-ca90-4dcf-ae73-050ff650abef admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.490312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307b6dc-ca90-4dcf-ae73-050ff650abef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.492086 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2307b6dc-ca90-4dcf-ae73-050ff650abef admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.496950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66697b24-fc35-4fb6-aa33-2ff2ac2086e9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.501176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e754e1d-1e91-42c7-8e68-61bb398c3af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.502046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e754e1d-1e91-42c7-8e68-61bb398c3af2 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.502963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e754e1d-1e91-42c7-8e68-61bb398c3af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.503376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e754e1d-1e91-42c7-8e68-61bb398c3af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.503772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 795/1593] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.504798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307b6dc-ca90-4dcf-ae73-050ff650abef admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.515348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2307b6dc-ca90-4dcf-ae73-050ff650abef admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.518012 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e754e1d-1e91-42c7-8e68-61bb398c3af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.519513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e754e1d-1e91-42c7-8e68-61bb398c3af2 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.524349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdf6b0-4883-4290-a954-15cbefdfc29f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.525602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdf6b0-4883-4290-a954-15cbefdfc29f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.526059 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdf6b0-4883-4290-a954-15cbefdfc29f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.526451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdf6b0-4883-4290-a954-15cbefdfc29f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.527444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdf6b0-4883-4290-a954-15cbefdfc29f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.530385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdf6b0-4883-4290-a954-15cbefdfc29f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.534688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 799/1594] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:09.536142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e754e1d-1e91-42c7-8e68-61bb398c3af2 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.548894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e754e1d-1e91-42c7-8e68-61bb398c3af2 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.558964 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d996c-83ad-4ea7-aef4-2570fa826408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.560144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d996c-83ad-4ea7-aef4-2570fa826408 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.560465 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d996c-83ad-4ea7-aef4-2570fa826408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.560741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8d996c-83ad-4ea7-aef4-2570fa826408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.561917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d996c-83ad-4ea7-aef4-2570fa826408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.564048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8d996c-83ad-4ea7-aef4-2570fa826408 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.565828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 796/1595] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.567905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdf6b0-4883-4290-a954-15cbefdfc29f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.582600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 800/1596] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.582600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d996c-83ad-4ea7-aef4-2570fa826408 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.587083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdf6b0-4883-4290-a954-15cbefdfc29f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.593716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ef0e0-baa3-42d0-a70f-79ee716622fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.594795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ef0e0-baa3-42d0-a70f-79ee716622fc admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.594795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ef0e0-baa3-42d0-a70f-79ee716622fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.595166 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522ef0e0-baa3-42d0-a70f-79ee716622fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.595954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ef0e0-baa3-42d0-a70f-79ee716622fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.598992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522ef0e0-baa3-42d0-a70f-79ee716622fc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.600027 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8d996c-83ad-4ea7-aef4-2570fa826408 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.606344 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b25caf-c72b-473f-b461-9c0ecf8145d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.609012 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b25caf-c72b-473f-b461-9c0ecf8145d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.609407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b25caf-c72b-473f-b461-9c0ecf8145d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.609875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b25caf-c72b-473f-b461-9c0ecf8145d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.610956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b25caf-c72b-473f-b461-9c0ecf8145d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.613048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b25caf-c72b-473f-b461-9c0ecf8145d7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.614008 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 797/1597] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.619768 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ef0e0-baa3-42d0-a70f-79ee716622fc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.635827 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 801/1598] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.635827 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b25caf-c72b-473f-b461-9c0ecf8145d7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.639772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522ef0e0-baa3-42d0-a70f-79ee716622fc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.647577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7915e81-d45d-42d9-8c52-0ffd4d543c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.648372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7915e81-d45d-42d9-8c52-0ffd4d543c72 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.648666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7915e81-d45d-42d9-8c52-0ffd4d543c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.648964 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7915e81-d45d-42d9-8c52-0ffd4d543c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.649524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b25caf-c72b-473f-b461-9c0ecf8145d7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.649834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7915e81-d45d-42d9-8c52-0ffd4d543c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.652083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7915e81-d45d-42d9-8c52-0ffd4d543c72 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.656920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78531cbe-bd6e-4f41-a1cb-6099d4c2f117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.658413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78531cbe-bd6e-4f41-a1cb-6099d4c2f117 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.659636 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78531cbe-bd6e-4f41-a1cb-6099d4c2f117 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.660308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78531cbe-bd6e-4f41-a1cb-6099d4c2f117 admin admin] PATH_INFO: `/v3/projects/74c80a6b1b314802a9b61b486583d8c3/users/c760993f65ba499c967de8cdb207035a/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.673559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 798/1599] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.677586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7915e81-d45d-42d9-8c52-0ffd4d543c72 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.688825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78531cbe-bd6e-4f41-a1cb-6099d4c2f117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74c80a6b1b314802a9b61b486583d8c3, user_id=c760993f65ba499c967de8cdb207035a, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.692900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78531cbe-bd6e-4f41-a1cb-6099d4c2f117 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.698297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7915e81-d45d-42d9-8c52-0ffd4d543c72 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.704158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ae4c4-db69-4ee6-82e8-f2b1f756aa01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.705460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ae4c4-db69-4ee6-82e8-f2b1f756aa01 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.705460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ae4c4-db69-4ee6-82e8-f2b1f756aa01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.706363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978ae4c4-db69-4ee6-82e8-f2b1f756aa01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.707933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ae4c4-db69-4ee6-82e8-f2b1f756aa01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.709847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978ae4c4-db69-4ee6-82e8-f2b1f756aa01 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.722541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 802/1600] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:09 2022] PUT /identity/v3/projects/74c80a6b1b314802a9b61b486583d8c3/users/c760993f65ba499c967de8cdb207035a/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:09.723622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78531cbe-bd6e-4f41-a1cb-6099d4c2f117 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.729087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 799/1601] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.730969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ae4c4-db69-4ee6-82e8-f2b1f756aa01 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.747339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978ae4c4-db69-4ee6-82e8-f2b1f756aa01 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.805492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78531cbe-bd6e-4f41-a1cb-6099d4c2f117 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.808414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:09.811960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a389f8a-3aa3-4833-80b7-a519f2884294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.813081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a389f8a-3aa3-4833-80b7-a519f2884294 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.813543 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a389f8a-3aa3-4833-80b7-a519f2884294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.813844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a389f8a-3aa3-4833-80b7-a519f2884294 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.815610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a389f8a-3aa3-4833-80b7-a519f2884294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.818647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a389f8a-3aa3-4833-80b7-a519f2884294 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.832788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 803/1602] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:09.833608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a389f8a-3aa3-4833-80b7-a519f2884294 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.836808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17dcfa-50da-46b0-886d-14bc1bd030b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.838241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17dcfa-50da-46b0-886d-14bc1bd030b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.838528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17dcfa-50da-46b0-886d-14bc1bd030b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.838879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a17dcfa-50da-46b0-886d-14bc1bd030b7 admin admin] PATH_INFO: `/v3/projects/e93fc8565caa47a5b3128d76c3969127/users/0329c5da1b12499991e966170aeabbd9/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.851473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a389f8a-3aa3-4833-80b7-a519f2884294 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:09.856524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17dcfa-50da-46b0-886d-14bc1bd030b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e93fc8565caa47a5b3128d76c3969127, user_id=0329c5da1b12499991e966170aeabbd9, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.857672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913617d3-212b-49f0-ba6d-b533059ecd8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:09.858835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913617d3-212b-49f0-ba6d-b533059ecd8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.859330 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913617d3-212b-49f0-ba6d-b533059ecd8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.859692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a17dcfa-50da-46b0-886d-14bc1bd030b7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.859813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913617d3-212b-49f0-ba6d-b533059ecd8b admin admin] PATH_INFO: `/v3/projects/72095d9091a24165a010fa994b41933a/users/f5b2d1a1fe9a448e961bff1439f1d467/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.885489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 800/1603] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:09 2022] PUT /identity/v3/projects/e93fc8565caa47a5b3128d76c3969127/users/0329c5da1b12499991e966170aeabbd9/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:09.886780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17dcfa-50da-46b0-886d-14bc1bd030b7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:09.890593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913617d3-212b-49f0-ba6d-b533059ecd8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=72095d9091a24165a010fa994b41933a, user_id=f5b2d1a1fe9a448e961bff1439f1d467, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:09.894727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913617d3-212b-49f0-ba6d-b533059ecd8b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:09.930981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 804/1604] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:09 2022] PUT /identity/v3/projects/72095d9091a24165a010fa994b41933a/users/f5b2d1a1fe9a448e961bff1439f1d467/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:09.930981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d8a22-8b65-4600-962d-5e43cd59cf53 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:09.930981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d8a22-8b65-4600-962d-5e43cd59cf53 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:09.930981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d8a22-8b65-4600-962d-5e43cd59cf53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:09.952858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-702d8a22-8b65-4600-962d-5e43cd59cf53 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:09.954438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-702d8a22-8b65-4600-962d-5e43cd59cf53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:09.994916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:09.995401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a17dcfa-50da-46b0-886d-14bc1bd030b7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:10.040176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 805/1605] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:10.041139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d8a22-8b65-4600-962d-5e43cd59cf53 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:10.057848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702d8a22-8b65-4600-962d-5e43cd59cf53 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:10.069391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990ef8b-bd0a-4327-8aaa-642ada2b2e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:10.070903 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990ef8b-bd0a-4327-8aaa-642ada2b2e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.071365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990ef8b-bd0a-4327-8aaa-642ada2b2e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.071862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990ef8b-bd0a-4327-8aaa-642ada2b2e92 admin admin] PATH_INFO: `/v3/projects/d75bcce87627476fa9be86774325d62d/users/6161475cb848468e92eed1a657e2c835/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.085857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a16bcf-a49d-4acd-8545-5f3fa119f7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:10.086785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a16bcf-a49d-4acd-8545-5f3fa119f7d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.087102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a16bcf-a49d-4acd-8545-5f3fa119f7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.087341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a16bcf-a49d-4acd-8545-5f3fa119f7d8 admin admin] PATH_INFO: `/v3/projects/d44719e63bb947bdb13e9309d1576521/users/83bbc8ad40b44661b526511ecbc51027/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.097479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990ef8b-bd0a-4327-8aaa-642ada2b2e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d75bcce87627476fa9be86774325d62d, user_id=6161475cb848468e92eed1a657e2c835, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:10.100323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990ef8b-bd0a-4327-8aaa-642ada2b2e92 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:10.105888 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a16bcf-a49d-4acd-8545-5f3fa119f7d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d44719e63bb947bdb13e9309d1576521, user_id=83bbc8ad40b44661b526511ecbc51027, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:10.109012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a16bcf-a49d-4acd-8545-5f3fa119f7d8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:10.119122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 806/1606] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:10 2022] PUT /identity/v3/projects/d75bcce87627476fa9be86774325d62d/users/6161475cb848468e92eed1a657e2c835/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:10.120229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990ef8b-bd0a-4327-8aaa-642ada2b2e92 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:10.131048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 801/1607] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:09 2022] PUT /identity/v3/projects/d44719e63bb947bdb13e9309d1576521/users/83bbc8ad40b44661b526511ecbc51027/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:10.131048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c454f-7fc1-4cce-8bd9-9125e92fdaf4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.131048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c454f-7fc1-4cce-8bd9-9125e92fdaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.131048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8c454f-7fc1-4cce-8bd9-9125e92fdaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.168305 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ea8c454f-7fc1-4cce-8bd9-9125e92fdaf4 None None] MFA Rules not processed for user `0329c5da1b12499991e966170aeabbd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:10.169738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ea8c454f-7fc1-4cce-8bd9-9125e92fdaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:10.210286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990ef8b-bd0a-4327-8aaa-642ada2b2e92 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:10.243129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:10.308562 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ebfd-a8eb-46eb-a235-9c852b69abee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:10.309820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ebfd-a8eb-46eb-a235-9c852b69abee admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.310114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ebfd-a8eb-46eb-a235-9c852b69abee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.310542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e1ebfd-a8eb-46eb-a235-9c852b69abee admin admin] PATH_INFO: `/v3/projects/2c27fe465e6743e19feec424e6e9f3d1/users/8f448776d1074e22bfe6cfbe05e01db7/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.318783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 802/1608] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:10.319918 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b0ccc-4d49-49df-a05c-230621b10502 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.320219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b0ccc-4d49-49df-a05c-230621b10502 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.320501 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8b0ccc-4d49-49df-a05c-230621b10502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.340189 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1ebfd-a8eb-46eb-a235-9c852b69abee admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c27fe465e6743e19feec424e6e9f3d1, user_id=8f448776d1074e22bfe6cfbe05e01db7, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:10.343217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e1ebfd-a8eb-46eb-a235-9c852b69abee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:10.372356 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cc8b0ccc-4d49-49df-a05c-230621b10502 None None] MFA Rules not processed for user `f5b2d1a1fe9a448e961bff1439f1d467`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:10.373858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cc8b0ccc-4d49-49df-a05c-230621b10502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:10.376882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 807/1609] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:10 2022] PUT /identity/v3/projects/2c27fe465e6743e19feec424e6e9f3d1/users/8f448776d1074e22bfe6cfbe05e01db7/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:10.380419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ebfd-a8eb-46eb-a235-9c852b69abee admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:10.446261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:10.462422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e1ebfd-a8eb-46eb-a235-9c852b69abee admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:10.466966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9e5311-73b1-403c-a5c6-5d2a28eb59f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:10.467850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e5311-73b1-403c-a5c6-5d2a28eb59f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.468288 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e5311-73b1-403c-a5c6-5d2a28eb59f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.468816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9e5311-73b1-403c-a5c6-5d2a28eb59f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.477423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 803/1610] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:10.477423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4bcec-2e6b-487c-91c9-cd5645bbfc03 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.477423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4bcec-2e6b-487c-91c9-cd5645bbfc03 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.477423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e4bcec-2e6b-487c-91c9-cd5645bbfc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.512722 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-41e4bcec-2e6b-487c-91c9-cd5645bbfc03 None None] MFA Rules not processed for user `6161475cb848468e92eed1a657e2c835`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:10.515445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-41e4bcec-2e6b-487c-91c9-cd5645bbfc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:10.552104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e5311-73b1-403c-a5c6-5d2a28eb59f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:10.553864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9e5311-73b1-403c-a5c6-5d2a28eb59f0 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:10.565110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 808/1611] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:10.566817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d73c6-57ee-42db-b265-fc502538fc66 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.567114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d73c6-57ee-42db-b265-fc502538fc66 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.567399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d73c6-57ee-42db-b265-fc502538fc66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.586382 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6f1d73c6-57ee-42db-b265-fc502538fc66 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:10.587649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6f1d73c6-57ee-42db-b265-fc502538fc66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:10.602456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 809/1612] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:10.603407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d73c6-57ee-42db-b265-fc502538fc66 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:10.615921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d73c6-57ee-42db-b265-fc502538fc66 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:10.621290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b050387-cf94-47f1-b86e-89feb6d48acd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:10.622030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b050387-cf94-47f1-b86e-89feb6d48acd admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.622513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b050387-cf94-47f1-b86e-89feb6d48acd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.622871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b050387-cf94-47f1-b86e-89feb6d48acd admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.623643 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b050387-cf94-47f1-b86e-89feb6d48acd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:10.625852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b050387-cf94-47f1-b86e-89feb6d48acd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:10.658172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 804/1613] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:10 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:10.660608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a963ab7-fa94-41a7-bbce-4f6685495b72 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.660915 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a963ab7-fa94-41a7-bbce-4f6685495b72 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.661195 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a963ab7-fa94-41a7-bbce-4f6685495b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.682922 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 810/1614] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:10 2022] POST /identity/v3/projects => generated 381 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:10.684415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b050387-cf94-47f1-b86e-89feb6d48acd admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:10.696359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9a963ab7-fa94-41a7-bbce-4f6685495b72 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:10.700293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9a963ab7-fa94-41a7-bbce-4f6685495b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:10.874193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 805/1615] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:10.875237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a963ab7-fa94-41a7-bbce-4f6685495b72 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:10.882949 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:10.933725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b050387-cf94-47f1-b86e-89feb6d48acd admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:10.960121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04eea5-7c26-4ae9-a2fd-031d27987166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:10.961320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04eea5-7c26-4ae9-a2fd-031d27987166 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:10.961805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04eea5-7c26-4ae9-a2fd-031d27987166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:10.962173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e04eea5-7c26-4ae9-a2fd-031d27987166 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:10.963069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04eea5-7c26-4ae9-a2fd-031d27987166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:10.965236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e04eea5-7c26-4ae9-a2fd-031d27987166 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:11.032386 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a963ab7-fa94-41a7-bbce-4f6685495b72 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:11.048062 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748a1ccd-a41b-488a-a072-fcf2ceeb537e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:11.050380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a1ccd-a41b-488a-a072-fcf2ceeb537e service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.051117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a1ccd-a41b-488a-a072-fcf2ceeb537e service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.052211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748a1ccd-a41b-488a-a072-fcf2ceeb537e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.066846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 811/1616] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:10 2022] POST /identity/v3/projects => generated 363 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:11.068580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04eea5-7c26-4ae9-a2fd-031d27987166 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:11.085414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a1ccd-a41b-488a-a072-fcf2ceeb537e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:11.088370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748a1ccd-a41b-488a-a072-fcf2ceeb537e service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:11.116687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 806/1617] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:11.124371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ea486-7586-464a-873d-d98c5f0ac7f5 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.124970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ea486-7586-464a-873d-d98c5f0ac7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.125459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53ea486-7586-464a-873d-d98c5f0ac7f5 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.131679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 807/1618] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jul 12 13:27:11 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:11.132993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ea486-7586-464a-873d-d98c5f0ac7f5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:11.258759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:11.291672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e04eea5-7c26-4ae9-a2fd-031d27987166 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:11.304007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87ba9-d0cc-4b7c-9348-954bd7ec499f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:11.305697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87ba9-d0cc-4b7c-9348-954bd7ec499f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.306291 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87ba9-d0cc-4b7c-9348-954bd7ec499f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.307095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87ba9-d0cc-4b7c-9348-954bd7ec499f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.307928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53ea486-7586-464a-873d-d98c5f0ac7f5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:11.308516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf87ba9-d0cc-4b7c-9348-954bd7ec499f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:11.314213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf87ba9-d0cc-4b7c-9348-954bd7ec499f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:11.319447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d7a068-d995-4fdc-a1e4-af55164ce0c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:11.320921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7a068-d995-4fdc-a1e4-af55164ce0c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.321469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7a068-d995-4fdc-a1e4-af55164ce0c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.321849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d7a068-d995-4fdc-a1e4-af55164ce0c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.344543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7a068-d995-4fdc-a1e4-af55164ce0c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:11.354394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d7a068-d995-4fdc-a1e4-af55164ce0c1 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:11.394554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 812/1619] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:11 2022] POST /identity/v3/users => generated 297 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:11.397071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87ba9-d0cc-4b7c-9348-954bd7ec499f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:11.431541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 808/1620] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:11.434905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce861f4-a6bf-4035-bcc3-21e2bcec0e1a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.435873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce861f4-a6bf-4035-bcc3-21e2bcec0e1a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.436406 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce861f4-a6bf-4035-bcc3-21e2bcec0e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.438719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87ba9-d0cc-4b7c-9348-954bd7ec499f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:11.453319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153379df-8824-4172-a479-4f1827fff9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:11.454206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153379df-8824-4172-a479-4f1827fff9ea admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.454590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153379df-8824-4172-a479-4f1827fff9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.455001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153379df-8824-4172-a479-4f1827fff9ea admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.457881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153379df-8824-4172-a479-4f1827fff9ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:11.462610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153379df-8824-4172-a479-4f1827fff9ea admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:11.516838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 813/1621] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:11 2022] POST /identity/v3/users => generated 288 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:11.517618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153379df-8824-4172-a479-4f1827fff9ea admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:11.539720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153379df-8824-4172-a479-4f1827fff9ea admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:11.550499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037c580-e6d8-4bde-8f84-9e1704e23837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:11.553324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037c580-e6d8-4bde-8f84-9e1704e23837 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.553779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037c580-e6d8-4bde-8f84-9e1704e23837 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.554991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5037c580-e6d8-4bde-8f84-9e1704e23837 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.555863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037c580-e6d8-4bde-8f84-9e1704e23837 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:11.558811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5037c580-e6d8-4bde-8f84-9e1704e23837 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:11.573650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fce861f4-a6bf-4035-bcc3-21e2bcec0e1a None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:11.575091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fce861f4-a6bf-4035-bcc3-21e2bcec0e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:11.621630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 814/1622] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:11.626457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037c580-e6d8-4bde-8f84-9e1704e23837 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:11.656970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5037c580-e6d8-4bde-8f84-9e1704e23837 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:11.671488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e575b-e75f-4ea7-b311-3479679419cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:11.673718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e575b-e75f-4ea7-b311-3479679419cf admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.675571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e575b-e75f-4ea7-b311-3479679419cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.676185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e575b-e75f-4ea7-b311-3479679419cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.678525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e575b-e75f-4ea7-b311-3479679419cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:11.687965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e575b-e75f-4ea7-b311-3479679419cf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:11.729357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 815/1623] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:11.729357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e575b-e75f-4ea7-b311-3479679419cf admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:11.759313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 809/1624] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jul 12 13:27:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:11.762117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce861f4-a6bf-4035-bcc3-21e2bcec0e1a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:11.764629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e575b-e75f-4ea7-b311-3479679419cf admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:11.776628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361536d-95bc-46d4-95de-76bce2918b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:11.778889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361536d-95bc-46d4-95de-76bce2918b28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.779522 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361536d-95bc-46d4-95de-76bce2918b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.780590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d361536d-95bc-46d4-95de-76bce2918b28 admin admin] PATH_INFO: `/v3/projects/a6fb4f8909554b928018143e2017c73f/users/6be665c86ead4d5ab71902e908d53747/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.791476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce861f4-a6bf-4035-bcc3-21e2bcec0e1a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:11.803416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dd81f-9f03-4e85-a8bb-aed1d0de92d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:11.806294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dd81f-9f03-4e85-a8bb-aed1d0de92d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:11.807930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dd81f-9f03-4e85-a8bb-aed1d0de92d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:11.808329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489dd81f-9f03-4e85-a8bb-aed1d0de92d4 admin admin] PATH_INFO: `/v3/projects/28f68c2a85b34694a19e6a00590f3ec7/users/36a206d8864e41d99ebba2430585c354/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:11.831162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361536d-95bc-46d4-95de-76bce2918b28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fb4f8909554b928018143e2017c73f, user_id=6be665c86ead4d5ab71902e908d53747, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:11.845192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d361536d-95bc-46d4-95de-76bce2918b28 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:11.864994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dd81f-9f03-4e85-a8bb-aed1d0de92d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f68c2a85b34694a19e6a00590f3ec7, user_id=36a206d8864e41d99ebba2430585c354, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:11.875362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 816/1625] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:11 2022] PUT /identity/v3/projects/a6fb4f8909554b928018143e2017c73f/users/6be665c86ead4d5ab71902e908d53747/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:11.876667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361536d-95bc-46d4-95de-76bce2918b28 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:11.892060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489dd81f-9f03-4e85-a8bb-aed1d0de92d4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:11.967350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 810/1626] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:11 2022] PUT /identity/v3/projects/28f68c2a85b34694a19e6a00590f3ec7/users/36a206d8864e41d99ebba2430585c354/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:11.967350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dd81f-9f03-4e85-a8bb-aed1d0de92d4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:11.981565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:12.054933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d361536d-95bc-46d4-95de-76bce2918b28 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:12.107826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489dd81f-9f03-4e85-a8bb-aed1d0de92d4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:12.116078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e807ae-daa8-4409-a217-53c866d1e6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:12.116838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e807ae-daa8-4409-a217-53c866d1e6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:12.117240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e807ae-daa8-4409-a217-53c866d1e6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:12.117674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e807ae-daa8-4409-a217-53c866d1e6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:12.119167 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e807ae-daa8-4409-a217-53c866d1e6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:12.123111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e807ae-daa8-4409-a217-53c866d1e6b1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:12.141904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 811/1627] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:12.143720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e807ae-daa8-4409-a217-53c866d1e6b1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:12.158101 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e807ae-daa8-4409-a217-53c866d1e6b1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:12.178115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988a20a-7f57-4969-a94f-db7db8375999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:12.182802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988a20a-7f57-4969-a94f-db7db8375999 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:12.183387 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988a20a-7f57-4969-a94f-db7db8375999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:12.184573 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5988a20a-7f57-4969-a94f-db7db8375999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:12.186441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988a20a-7f57-4969-a94f-db7db8375999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:12.191747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5988a20a-7f57-4969-a94f-db7db8375999 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:12.216651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 812/1628] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:12.218350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988a20a-7f57-4969-a94f-db7db8375999 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:12.234816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b66885-0ce8-4cb0-a5f2-cf7211028d33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:12.235665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b66885-0ce8-4cb0-a5f2-cf7211028d33 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:12.237647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b66885-0ce8-4cb0-a5f2-cf7211028d33 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:12.238071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b66885-0ce8-4cb0-a5f2-cf7211028d33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:12.241856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5988a20a-7f57-4969-a94f-db7db8375999 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:12.261415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce98c5f-cd60-43a3-872f-cc93d6e60f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:12.264675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:12.266587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce98c5f-cd60-43a3-872f-cc93d6e60f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:12.270472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce98c5f-cd60-43a3-872f-cc93d6e60f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:12.271316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce98c5f-cd60-43a3-872f-cc93d6e60f62 admin admin] PATH_INFO: `/v3/projects/a6fb4f8909554b928018143e2017c73f/users/6be665c86ead4d5ab71902e908d53747/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:12.278639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce98c5f-cd60-43a3-872f-cc93d6e60f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fb4f8909554b928018143e2017c73f, user_id=6be665c86ead4d5ab71902e908d53747, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:12.320605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce98c5f-cd60-43a3-872f-cc93d6e60f62 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:12.361410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 813/1629] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:12 2022] PUT /identity/v3/projects/a6fb4f8909554b928018143e2017c73f/users/6be665c86ead4d5ab71902e908d53747/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:12.368410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce98c5f-cd60-43a3-872f-cc93d6e60f62 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:12.393055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b66885-0ce8-4cb0-a5f2-cf7211028d33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:12.395763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b66885-0ce8-4cb0-a5f2-cf7211028d33 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:12.421755 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 817/1630] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:12.423262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5d510-fc7e-47c6-8640-dd9341a85f3a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:12.423742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5d510-fc7e-47c6-8640-dd9341a85f3a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:12.424294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb5d510-fc7e-47c6-8640-dd9341a85f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:12.504696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ddb5d510-fc7e-47c6-8640-dd9341a85f3a None None] MFA Rules not processed for user `6be665c86ead4d5ab71902e908d53747`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:12.508185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ddb5d510-fc7e-47c6-8640-dd9341a85f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:12.514925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce98c5f-cd60-43a3-872f-cc93d6e60f62 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:12.522472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9945d5-66f7-4b9b-ac0a-1b18bf77a72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:12.526205 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9945d5-66f7-4b9b-ac0a-1b18bf77a72e admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:12.526558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9945d5-66f7-4b9b-ac0a-1b18bf77a72e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:12.526842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9945d5-66f7-4b9b-ac0a-1b18bf77a72e admin admin] PATH_INFO: `/v3/projects/28f68c2a85b34694a19e6a00590f3ec7/users/36a206d8864e41d99ebba2430585c354/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:12.529750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9945d5-66f7-4b9b-ac0a-1b18bf77a72e admin admin] RBAC: Authorizing `identity:create_grant(project_id=28f68c2a85b34694a19e6a00590f3ec7, user_id=36a206d8864e41d99ebba2430585c354, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:12.534890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9945d5-66f7-4b9b-ac0a-1b18bf77a72e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:12.551549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 814/1631] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:12 2022] PUT /identity/v3/projects/28f68c2a85b34694a19e6a00590f3ec7/users/36a206d8864e41d99ebba2430585c354/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:12.562788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c24707-a0c2-4bd9-bf4a-17e46ec8a3bc None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:12.563082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c24707-a0c2-4bd9-bf4a-17e46ec8a3bc None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:12.563358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c24707-a0c2-4bd9-bf4a-17e46ec8a3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:12.634146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-20c24707-a0c2-4bd9-bf4a-17e46ec8a3bc None None] MFA Rules not processed for user `36a206d8864e41d99ebba2430585c354`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:12.638371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-20c24707-a0c2-4bd9-bf4a-17e46ec8a3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:12.766089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 818/1632] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:12 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:12.782655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5d510-fc7e-47c6-8640-dd9341a85f3a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:12.860612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:12.902432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 815/1633] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:12 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:27:12.914097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c24707-a0c2-4bd9-bf4a-17e46ec8a3bc None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:12.925193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb5d510-fc7e-47c6-8640-dd9341a85f3a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:13.090099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cf781-95bc-4c84-a790-b9c6085c45ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:13.090985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cf781-95bc-4c84-a790-b9c6085c45ea service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:13.091562 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cf781-95bc-4c84-a790-b9c6085c45ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:13.092232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326cf781-95bc-4c84-a790-b9c6085c45ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:13.105094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:13.108818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c24707-a0c2-4bd9-bf4a-17e46ec8a3bc None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:13.124237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf85674-d723-4708-adcd-a2454aa85d73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:13.124237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf85674-d723-4708-adcd-a2454aa85d73 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:13.124237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf85674-d723-4708-adcd-a2454aa85d73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:13.124237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf85674-d723-4708-adcd-a2454aa85d73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:13.143828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf85674-d723-4708-adcd-a2454aa85d73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:13.145530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf85674-d723-4708-adcd-a2454aa85d73 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:13.179155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 816/1634] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:12 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:13.271200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cf781-95bc-4c84-a790-b9c6085c45ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:13.281202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326cf781-95bc-4c84-a790-b9c6085c45ea service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:13.324804 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 819/1635] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:12 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:13.446463 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf85674-d723-4708-adcd-a2454aa85d73 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:13.460915 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf85674-d723-4708-adcd-a2454aa85d73 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:13.476372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22709cd-37ff-4566-8098-5cc9909701fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:13.477521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22709cd-37ff-4566-8098-5cc9909701fc service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:13.478128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22709cd-37ff-4566-8098-5cc9909701fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:13.478747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22709cd-37ff-4566-8098-5cc9909701fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:13.650128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cf781-95bc-4c84-a790-b9c6085c45ea service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:13.660486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22709cd-37ff-4566-8098-5cc9909701fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:13.662260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22709cd-37ff-4566-8098-5cc9909701fc service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:13.665886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326cf781-95bc-4c84-a790-b9c6085c45ea service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:13.675113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 817/1636] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:13 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:13.678030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319858cf-1eb6-41a3-bdf4-b2edd83b655f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:13.678781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319858cf-1eb6-41a3-bdf4-b2edd83b655f service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:13.679004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319858cf-1eb6-41a3-bdf4-b2edd83b655f service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:13.679282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319858cf-1eb6-41a3-bdf4-b2edd83b655f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:13.823563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ca3a4-97cd-4a9e-9c90-49c87f4e2d25 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:13.824255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ca3a4-97cd-4a9e-9c90-49c87f4e2d25 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:13.824703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40ca3a4-97cd-4a9e-9c90-49c87f4e2d25 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:13.826713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 818/1637] 10.222.0.23 () {54 vars in 961 bytes} [Tue Jul 12 13:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:27:13.845226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e85260-5169-48f2-8145-72342525c234 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:13.845653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e85260-5169-48f2-8145-72342525c234 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:13.845808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e85260-5169-48f2-8145-72342525c234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:13.938510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-94e85260-5169-48f2-8145-72342525c234 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:13.941229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-94e85260-5169-48f2-8145-72342525c234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:13.952753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319858cf-1eb6-41a3-bdf4-b2edd83b655f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:13.956671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319858cf-1eb6-41a3-bdf4-b2edd83b655f service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:13.996345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 820/1638] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:27:14.174951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 819/1639] 10.222.0.23 () {58 vars in 1058 bytes} [Tue Jul 12 13:27:13 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:14.182003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b4090-c2ec-403b-9c99-60305a2f6947 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:14.182305 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b4090-c2ec-403b-9c99-60305a2f6947 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:14.183607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b4090-c2ec-403b-9c99-60305a2f6947 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:14.185542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 821/1640] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jul 12 13:27:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:27:14.198165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a6ff8-215c-4d32-8d1e-b9a6449b1207 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:14.198538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a6ff8-215c-4d32-8d1e-b9a6449b1207 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:14.198716 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9a6ff8-215c-4d32-8d1e-b9a6449b1207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:14.271288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9f9a6ff8-215c-4d32-8d1e-b9a6449b1207 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:14.275356 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9f9a6ff8-215c-4d32-8d1e-b9a6449b1207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:14.307395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 820/1641] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:27:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:14.364459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b4090-c2ec-403b-9c99-60305a2f6947 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:14.387969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b4090-c2ec-403b-9c99-60305a2f6947 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:14.399065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f45d2f-7181-4da3-9151-f435db4856d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:14.399804 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f45d2f-7181-4da3-9151-f435db4856d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:14.400243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f45d2f-7181-4da3-9151-f435db4856d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:14.400676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f45d2f-7181-4da3-9151-f435db4856d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:14.423193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f45d2f-7181-4da3-9151-f435db4856d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:14.428928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f45d2f-7181-4da3-9151-f435db4856d0 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:14.454368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 822/1642] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:14.712558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bee8d-a3ff-4097-a6dc-f09b5b48a7d0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:14.714207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bee8d-a3ff-4097-a6dc-f09b5b48a7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:14.714207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1bee8d-a3ff-4097-a6dc-f09b5b48a7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:14.765690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc558a-0c6a-4158-8b95-ba588893f3c6 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:14.766260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc558a-0c6a-4158-8b95-ba588893f3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:14.766657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc558a-0c6a-4158-8b95-ba588893f3c6 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:14.769489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 823/1643] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jul 12 13:27:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:27:14.773805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-da1bee8d-a3ff-4097-a6dc-f09b5b48a7d0 None None] MFA Rules not processed for user `c760993f65ba499c967de8cdb207035a`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:14.776542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-da1bee8d-a3ff-4097-a6dc-f09b5b48a7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:14.790824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:14.797262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6982c0-66ce-4de6-b225-d532e7561286 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:14.797708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6982c0-66ce-4de6-b225-d532e7561286 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:14.798091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6982c0-66ce-4de6-b225-d532e7561286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:14.840674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8d6982c0-66ce-4de6-b225-d532e7561286 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:14.855011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8d6982c0-66ce-4de6-b225-d532e7561286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:14.883847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 824/1644] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:27:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:27:14.907397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6982c0-66ce-4de6-b225-d532e7561286 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:14.921315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 821/1645] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:14.926086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6982c0-66ce-4de6-b225-d532e7561286 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:14.926816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bee8d-a3ff-4097-a6dc-f09b5b48a7d0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:14.938398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e68310-8edb-4b2d-b933-239063e662b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:14.939187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e68310-8edb-4b2d-b933-239063e662b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:14.939551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e68310-8edb-4b2d-b933-239063e662b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:14.939991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e68310-8edb-4b2d-b933-239063e662b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:14.960703 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e68310-8edb-4b2d-b933-239063e662b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:14.962198 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e68310-8edb-4b2d-b933-239063e662b2 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:14.989156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 825/1646] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:15.069726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1bee8d-a3ff-4097-a6dc-f09b5b48a7d0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:15.101658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a974c-e0d4-4e74-b889-8cdf132f6ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:15.104618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a974c-e0d4-4e74-b889-8cdf132f6ffd admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:15.105866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a974c-e0d4-4e74-b889-8cdf132f6ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:15.108615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8a974c-e0d4-4e74-b889-8cdf132f6ffd admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:15.109865 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a974c-e0d4-4e74-b889-8cdf132f6ffd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:15.120070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8a974c-e0d4-4e74-b889-8cdf132f6ffd admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:15.201272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 822/1647] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:14 2022] POST /identity/v3/projects => generated 403 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:15.206183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e68310-8edb-4b2d-b933-239063e662b2 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:15.232352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e68310-8edb-4b2d-b933-239063e662b2 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:15.295726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a974c-e0d4-4e74-b889-8cdf132f6ffd admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:15.421400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8a974c-e0d4-4e74-b889-8cdf132f6ffd admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:15.425844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca04c33-e9e4-494e-83c9-5a70b00779c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:15.427075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca04c33-e9e4-494e-83c9-5a70b00779c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:15.427459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca04c33-e9e4-494e-83c9-5a70b00779c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:15.427852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca04c33-e9e4-494e-83c9-5a70b00779c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:15.428797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca04c33-e9e4-494e-83c9-5a70b00779c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:15.431288 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca04c33-e9e4-494e-83c9-5a70b00779c4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:15.438566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:15.447620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:15.453667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:15.454052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:15.456337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] PATH_INFO: `/v3/users/6161475cb848468e92eed1a657e2c835` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:15.460283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6161475cb848468e92eed1a657e2c835)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:15.461465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: WARNING py.warnings [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:27:15.461465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: warnings.warn(msg) Jul 12 13:27:15.461465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: Jul 12 13:27:15.470069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 826/1648] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:15 2022] POST /identity/v3/users => generated 308 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:15.471897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca04c33-e9e4-494e-83c9-5a70b00779c4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:15.474930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:15.498469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca04c33-e9e4-494e-83c9-5a70b00779c4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:15.531837 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83b9dd-c6e6-4734-8c45-5fad2ad07d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:15.536168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83b9dd-c6e6-4734-8c45-5fad2ad07d47 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:15.536694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83b9dd-c6e6-4734-8c45-5fad2ad07d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:15.537923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83b9dd-c6e6-4734-8c45-5fad2ad07d47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:15.541985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83b9dd-c6e6-4734-8c45-5fad2ad07d47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:15.551530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83b9dd-c6e6-4734-8c45-5fad2ad07d47 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:15.591114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 827/1649] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:27:15.591114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83b9dd-c6e6-4734-8c45-5fad2ad07d47 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:15.617459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83b9dd-c6e6-4734-8c45-5fad2ad07d47 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:15.657772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38b038-2417-4c9f-9b14-0dd5a4fffbf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:15.660767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38b038-2417-4c9f-9b14-0dd5a4fffbf8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:15.663102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38b038-2417-4c9f-9b14-0dd5a4fffbf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:15.664217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d38b038-2417-4c9f-9b14-0dd5a4fffbf8 admin admin] PATH_INFO: `/v3/projects/3e93280502854638ad841ef17953ad6c/users/cb5f426772e747a79144cfde79919fe3/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:15.688810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:15.717820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38b038-2417-4c9f-9b14-0dd5a4fffbf8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e93280502854638ad841ef17953ad6c, user_id=cb5f426772e747a79144cfde79919fe3, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:15.719932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6161475cb848468e92eed1a657e2c835'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:15.726534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d38b038-2417-4c9f-9b14-0dd5a4fffbf8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:15.762686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6161475cb848468e92eed1a657e2c835'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:15.764982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6161475cb848468e92eed1a657e2c835'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:15.771382 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 828/1650] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:15 2022] PUT /identity/v3/projects/3e93280502854638ad841ef17953ad6c/users/cb5f426772e747a79144cfde79919fe3/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:15.772978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38b038-2417-4c9f-9b14-0dd5a4fffbf8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:15.786153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6161475cb848468e92eed1a657e2c835'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:15.847794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4e2c9282-b050-4c98-91a8-a78d3651135a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6161475cb848468e92eed1a657e2c835'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:15.869171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 823/1651] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:15 2022] DELETE /identity/v3/users/6161475cb848468e92eed1a657e2c835 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:15.874371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cfb9c-4b38-49ff-bccd-fd84236d6283 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:15.876672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cfb9c-4b38-49ff-bccd-fd84236d6283 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:15.877160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cfb9c-4b38-49ff-bccd-fd84236d6283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:15.936963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-426cfb9c-4b38-49ff-bccd-fd84236d6283 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:15.942664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-426cfb9c-4b38-49ff-bccd-fd84236d6283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:15.977986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d38b038-2417-4c9f-9b14-0dd5a4fffbf8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:16.135050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 824/1652] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:16.147391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bb724-4bb1-4c3a-9354-347b7f0913d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:16.148424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bb724-4bb1-4c3a-9354-347b7f0913d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:16.149093 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bb724-4bb1-4c3a-9354-347b7f0913d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:16.149665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bb724-4bb1-4c3a-9354-347b7f0913d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:16.150684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bb724-4bb1-4c3a-9354-347b7f0913d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:16.152921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bb724-4bb1-4c3a-9354-347b7f0913d2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:16.181794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cfb9c-4b38-49ff-bccd-fd84236d6283 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:16.183857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-426cfb9c-4b38-49ff-bccd-fd84236d6283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:16.187895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 829/1653] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 416 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:16.192592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bb724-4bb1-4c3a-9354-347b7f0913d2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:16.196334 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8d8bb724-4bb1-4c3a-9354-347b7f0913d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:16.224222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bb724-4bb1-4c3a-9354-347b7f0913d2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:16.237836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33591ec-fe0e-4c53-b303-c8a5c3da1dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:16.239118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33591ec-fe0e-4c53-b303-c8a5c3da1dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:16.240004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33591ec-fe0e-4c53-b303-c8a5c3da1dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:16.240460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33591ec-fe0e-4c53-b303-c8a5c3da1dc2 admin admin] PATH_INFO: `/v3/projects/3e93280502854638ad841ef17953ad6c/users/cb5f426772e747a79144cfde79919fe3/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:16.243045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33591ec-fe0e-4c53-b303-c8a5c3da1dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e93280502854638ad841ef17953ad6c, user_id=cb5f426772e747a79144cfde79919fe3, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:16.251680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33591ec-fe0e-4c53-b303-c8a5c3da1dc2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:16.285407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 830/1654] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:16 2022] PUT /identity/v3/projects/3e93280502854638ad841ef17953ad6c/users/cb5f426772e747a79144cfde79919fe3/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:16.321335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cfb9c-4b38-49ff-bccd-fd84236d6283 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:16.430836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7958fb0a-e6b7-4f8d-a1b5-b3a9e434fb99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:16.433058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958fb0a-e6b7-4f8d-a1b5-b3a9e434fb99 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:16.434783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958fb0a-e6b7-4f8d-a1b5-b3a9e434fb99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:16.435180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7958fb0a-e6b7-4f8d-a1b5-b3a9e434fb99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:16.505154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:16.592373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958fb0a-e6b7-4f8d-a1b5-b3a9e434fb99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:16.607261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7958fb0a-e6b7-4f8d-a1b5-b3a9e434fb99 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:16.639970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 825/1655] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:27:18.123699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33591ec-fe0e-4c53-b303-c8a5c3da1dc2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:18.125857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d33591ec-fe0e-4c53-b303-c8a5c3da1dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:18.144962 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33591ec-fe0e-4c53-b303-c8a5c3da1dc2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:18.149916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:18.150794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:18.151076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:18.151347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] PATH_INFO: `/v3/users/f5b2d1a1fe9a448e961bff1439f1d467` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:18.152733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b2d1a1fe9a448e961bff1439f1d467)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:18.156025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:18.212532 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:18.217525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b2d1a1fe9a448e961bff1439f1d467'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:18.233910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b2d1a1fe9a448e961bff1439f1d467'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:18.235020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b2d1a1fe9a448e961bff1439f1d467'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:18.258680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b2d1a1fe9a448e961bff1439f1d467'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:18.278346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b2d1a1fe9a448e961bff1439f1d467'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:18.295698 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 831/1656] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:18 2022] DELETE /identity/v3/users/f5b2d1a1fe9a448e961bff1439f1d467 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:18.303412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b940a5-dd72-416a-823d-b35a34336f74 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:18.304317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b940a5-dd72-416a-823d-b35a34336f74 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:18.304317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b940a5-dd72-416a-823d-b35a34336f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:18.341779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-91b940a5-dd72-416a-823d-b35a34336f74 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:18.345473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-91b940a5-dd72-416a-823d-b35a34336f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:18.499235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 826/1657] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:18.542346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:18.544321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:18.666170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b31928-6ac1-4f8b-8a9f-452609653d26 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:18.672304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ce3067-2ff8-4370-b74d-4b96c7352e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:18.673448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce3067-2ff8-4370-b74d-4b96c7352e72 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:18.678194 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce3067-2ff8-4370-b74d-4b96c7352e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:18.678679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ce3067-2ff8-4370-b74d-4b96c7352e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:18.718835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce3067-2ff8-4370-b74d-4b96c7352e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:18.720317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ce3067-2ff8-4370-b74d-4b96c7352e72 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:18.755442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 832/1658] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:19.742162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b940a5-dd72-416a-823d-b35a34336f74 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:19.743637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-91b940a5-dd72-416a-823d-b35a34336f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:19.771508 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b940a5-dd72-416a-823d-b35a34336f74 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:19.791400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:19.792392 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:19.792875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:19.793107 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] PATH_INFO: `/v3/projects/d75bcce87627476fa9be86774325d62d` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:19.795858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d75bcce87627476fa9be86774325d62d)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:19.797957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:19.954886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd75bcce87627476fa9be86774325d62d', 'request_id': 'req-2d8a4a72-3745-4dd7-8082-e519cc23c461'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:19.981950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd75bcce87627476fa9be86774325d62d', 'request_id': 'req-2d8a4a72-3745-4dd7-8082-e519cc23c461'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:20.014555 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd75bcce87627476fa9be86774325d62d', 'request_id': 'req-2d8a4a72-3745-4dd7-8082-e519cc23c461'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:20.056514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] The token cache is being invalidate because project d75bcce87627476fa9be86774325d62d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:20.056858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:20.059186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2d8a4a72-3745-4dd7-8082-e519cc23c461 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:20.069505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 827/1659] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:19 2022] DELETE /identity/v3/projects/d75bcce87627476fa9be86774325d62d => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:20.071438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a18f42-2750-47b0-8a8c-1b6f203e317d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:20.071909 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a18f42-2750-47b0-8a8c-1b6f203e317d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:20.072300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a18f42-2750-47b0-8a8c-1b6f203e317d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:20.119195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f9a18f42-2750-47b0-8a8c-1b6f203e317d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:20.123507 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f9a18f42-2750-47b0-8a8c-1b6f203e317d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:20.334104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 833/1660] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:20.350769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695a318-c26e-45af-acb4-9d91e11064f6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:20.351110 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695a318-c26e-45af-acb4-9d91e11064f6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:20.351497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a695a318-c26e-45af-acb4-9d91e11064f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:20.389193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a695a318-c26e-45af-acb4-9d91e11064f6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:20.392544 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a695a318-c26e-45af-acb4-9d91e11064f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:20.427983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 828/1661] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:20.434508 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a18f42-2750-47b0-8a8c-1b6f203e317d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:20.461119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a18f42-2750-47b0-8a8c-1b6f203e317d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:20.489301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcb06f-792a-4530-b3b7-0d0d4851499c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:20.489991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcb06f-792a-4530-b3b7-0d0d4851499c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:20.490620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcb06f-792a-4530-b3b7-0d0d4851499c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:20.491389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fcb06f-792a-4530-b3b7-0d0d4851499c admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:20.491840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fcb06f-792a-4530-b3b7-0d0d4851499c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:20.492209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fcb06f-792a-4530-b3b7-0d0d4851499c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:20.496568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fcb06f-792a-4530-b3b7-0d0d4851499c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:20.557920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 834/1662] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:20.562752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695a318-c26e-45af-acb4-9d91e11064f6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:20.595567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a695a318-c26e-45af-acb4-9d91e11064f6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:20.618835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e83e61-722b-4061-b68e-1466c179bdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:20.619518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e83e61-722b-4061-b68e-1466c179bdee admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:20.619861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e83e61-722b-4061-b68e-1466c179bdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:20.620070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e83e61-722b-4061-b68e-1466c179bdee admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:20.620779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e83e61-722b-4061-b68e-1466c179bdee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:20.622969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e83e61-722b-4061-b68e-1466c179bdee admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:20.718299 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 829/1663] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:20 2022] POST /identity/v3/projects => generated 403 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:20.719707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcb06f-792a-4530-b3b7-0d0d4851499c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:20.745492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fcb06f-792a-4530-b3b7-0d0d4851499c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:20.765016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3348f-a6d7-490d-87f2-ed747aeb3bc9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:20.765907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3348f-a6d7-490d-87f2-ed747aeb3bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:20.766346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a3348f-a6d7-490d-87f2-ed747aeb3bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:20.839212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a8a3348f-a6d7-490d-87f2-ed747aeb3bc9 None None] MFA Rules not processed for user `cb5f426772e747a79144cfde79919fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:20.847562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a8a3348f-a6d7-490d-87f2-ed747aeb3bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:20.855025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:20.925061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:20.955486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63427562-61f5-4a2a-9f19-52a9e33d6474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:20.956402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63427562-61f5-4a2a-9f19-52a9e33d6474 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:20.956696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63427562-61f5-4a2a-9f19-52a9e33d6474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:20.956976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63427562-61f5-4a2a-9f19-52a9e33d6474 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:20.957715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63427562-61f5-4a2a-9f19-52a9e33d6474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:20.970357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63427562-61f5-4a2a-9f19-52a9e33d6474 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:21.035249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 835/1664] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:20 2022] POST /identity/v3/users => generated 302 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:21.037218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63427562-61f5-4a2a-9f19-52a9e33d6474 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:21.054834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63427562-61f5-4a2a-9f19-52a9e33d6474 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:21.073688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbdfc2-0d4b-4c66-b1dc-a8e858761f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:21.076059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbdfc2-0d4b-4c66-b1dc-a8e858761f5e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:21.076601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbdfc2-0d4b-4c66-b1dc-a8e858761f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:21.077079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fbdfc2-0d4b-4c66-b1dc-a8e858761f5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:21.080556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbdfc2-0d4b-4c66-b1dc-a8e858761f5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:21.098163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fbdfc2-0d4b-4c66-b1dc-a8e858761f5e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:21.117782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 830/1665] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:21.125585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3348f-a6d7-490d-87f2-ed747aeb3bc9 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:21.133319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 836/1666] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:27:21.151563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a3348f-a6d7-490d-87f2-ed747aeb3bc9 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:21.158005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe290aa-27b9-4e63-aca2-5810cc1a4f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:21.160129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe290aa-27b9-4e63-aca2-5810cc1a4f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:21.160637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe290aa-27b9-4e63-aca2-5810cc1a4f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:21.161254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe290aa-27b9-4e63-aca2-5810cc1a4f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:21.175733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe290aa-27b9-4e63-aca2-5810cc1a4f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:21.179329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe290aa-27b9-4e63-aca2-5810cc1a4f64 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:21.216307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbdfc2-0d4b-4c66-b1dc-a8e858761f5e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:21.223423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 831/1667] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:21.231603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fbdfc2-0d4b-4c66-b1dc-a8e858761f5e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:21.248438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6323e-4a25-46e1-bd87-ab7b2f941b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:21.249939 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6323e-4a25-46e1-bd87-ab7b2f941b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:21.254468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6323e-4a25-46e1-bd87-ab7b2f941b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:21.254924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f6323e-4a25-46e1-bd87-ab7b2f941b6a admin admin] PATH_INFO: `/v3/projects/730610bab7284e04bb14fa0752bc3289/users/7d6a4e54fe0f489f8e581062840b5077/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:21.296210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6323e-4a25-46e1-bd87-ab7b2f941b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=730610bab7284e04bb14fa0752bc3289, user_id=7d6a4e54fe0f489f8e581062840b5077, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:21.299813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f6323e-4a25-46e1-bd87-ab7b2f941b6a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:21.331338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 837/1668] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:21 2022] PUT /identity/v3/projects/730610bab7284e04bb14fa0752bc3289/users/7d6a4e54fe0f489f8e581062840b5077/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:21.342017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a686b-de0a-4ff2-93a1-76661c8039a9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:21.342515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a686b-de0a-4ff2-93a1-76661c8039a9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:21.342943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99a686b-de0a-4ff2-93a1-76661c8039a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:21.378377 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d99a686b-de0a-4ff2-93a1-76661c8039a9 None None] MFA Rules not processed for user `7d6a4e54fe0f489f8e581062840b5077`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:21.379631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d99a686b-de0a-4ff2-93a1-76661c8039a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:21.442725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6323e-4a25-46e1-bd87-ab7b2f941b6a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:21.444753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-18f6323e-4a25-46e1-bd87-ab7b2f941b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:21.652998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 832/1669] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:21.652998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc77921-9d86-4f0f-a7a8-d24b294a2f79 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:21.652998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc77921-9d86-4f0f-a7a8-d24b294a2f79 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:21.652998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc77921-9d86-4f0f-a7a8-d24b294a2f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:21.691003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f6323e-4a25-46e1-bd87-ab7b2f941b6a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:21.698930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:21.709336 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:21.710895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:21.711479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] PATH_INFO: `/v3/projects/72095d9091a24165a010fa994b41933a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:21.723522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3fc77921-9d86-4f0f-a7a8-d24b294a2f79 None None] MFA Rules not processed for user `f8de4128718d4837af77d427973d30be`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:21.724257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72095d9091a24165a010fa994b41933a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:21.724794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3fc77921-9d86-4f0f-a7a8-d24b294a2f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:21.726502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: WARNING py.warnings [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 12 13:27:21.726502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: warnings.warn(msg) Jul 12 13:27:21.726502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: Jul 12 13:27:21.729037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:21.855230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72095d9091a24165a010fa994b41933a', 'request_id': 'req-85c5e619-e945-4e1e-aa88-39307d3adab9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:21.874991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72095d9091a24165a010fa994b41933a', 'request_id': 'req-85c5e619-e945-4e1e-aa88-39307d3adab9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:21.894604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72095d9091a24165a010fa994b41933a', 'request_id': 'req-85c5e619-e945-4e1e-aa88-39307d3adab9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:21.953591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] The token cache is being invalidate because project 72095d9091a24165a010fa994b41933a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:21.953855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:21.957580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:21.973802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 838/1670] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:21 2022] DELETE /identity/v3/projects/72095d9091a24165a010fa994b41933a => generated 0 bytes in 528 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:21.973802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:21.977477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:21.991748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 833/1671] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:21.991748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc77921-9d86-4f0f-a7a8-d24b294a2f79 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:22.026997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc77921-9d86-4f0f-a7a8-d24b294a2f79 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:22.035877 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:22.155159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:22.210743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:22.211870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:22.212428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:22.212819 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] PATH_INFO: `/v3/users/36a206d8864e41d99ebba2430585c354` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:22.217394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] RBAC: Authorizing `identity:delete_user(user_id=36a206d8864e41d99ebba2430585c354)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:22.223016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c5e619-e945-4e1e-aa88-39307d3adab9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:22.228811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:22.245708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09dd8b-0b2f-4d22-a9f2-8a914be740d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:22.252825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09dd8b-0b2f-4d22-a9f2-8a914be740d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:22.253550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09dd8b-0b2f-4d22-a9f2-8a914be740d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:22.254063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b09dd8b-0b2f-4d22-a9f2-8a914be740d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:22.259011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09dd8b-0b2f-4d22-a9f2-8a914be740d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:22.261552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b09dd8b-0b2f-4d22-a9f2-8a914be740d8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:22.349483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 839/1672] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:21 2022] POST /identity/v3/projects => generated 405 bytes in 378 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:22.352106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09dd8b-0b2f-4d22-a9f2-8a914be740d8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:22.356736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0b09dd8b-0b2f-4d22-a9f2-8a914be740d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:22.369994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36a206d8864e41d99ebba2430585c354'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:22.393639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36a206d8864e41d99ebba2430585c354'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:22.394909 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36a206d8864e41d99ebba2430585c354'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:22.413823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36a206d8864e41d99ebba2430585c354'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:22.451680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36a206d8864e41d99ebba2430585c354'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:22.470932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 834/1673] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:21 2022] DELETE /identity/v3/users/36a206d8864e41d99ebba2430585c354 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:22.472772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:22.479658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:22.566467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b09dd8b-0b2f-4d22-a9f2-8a914be740d8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:22.690876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ae5e1a-5697-4080-8f63-639d2dd1a21b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:22.698985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c7ae9-7f4b-450f-a226-8d41faaa4e74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:22.699805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c7ae9-7f4b-450f-a226-8d41faaa4e74 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:22.700161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c7ae9-7f4b-450f-a226-8d41faaa4e74 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:22.701120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148c82f-f0ee-4531-b0b3-e84533d99e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:22.702174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885c7ae9-7f4b-450f-a226-8d41faaa4e74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:22.703115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148c82f-f0ee-4531-b0b3-e84533d99e03 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:22.704099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148c82f-f0ee-4531-b0b3-e84533d99e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:22.704617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-885c7ae9-7f4b-450f-a226-8d41faaa4e74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:22.705655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9148c82f-f0ee-4531-b0b3-e84533d99e03 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:22.708651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148c82f-f0ee-4531-b0b3-e84533d99e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:22.713364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9148c82f-f0ee-4531-b0b3-e84533d99e03 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:22.771277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 835/1674] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:22 2022] POST /identity/v3/users => generated 309 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:22.773340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148c82f-f0ee-4531-b0b3-e84533d99e03 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:22.802605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9148c82f-f0ee-4531-b0b3-e84533d99e03 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:22.824048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a52aeb7-f310-4f81-b37e-4c45ae6e4f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:22.825472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52aeb7-f310-4f81-b37e-4c45ae6e4f3e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:22.826659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52aeb7-f310-4f81-b37e-4c45ae6e4f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:22.827456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a52aeb7-f310-4f81-b37e-4c45ae6e4f3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:22.833204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52aeb7-f310-4f81-b37e-4c45ae6e4f3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:22.839309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a52aeb7-f310-4f81-b37e-4c45ae6e4f3e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:22.843908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c7ae9-7f4b-450f-a226-8d41faaa4e74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:22.847030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885c7ae9-7f4b-450f-a226-8d41faaa4e74 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:22.855368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:22.872082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 840/1675] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:22 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:22.880998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 836/1676] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:22.894233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c7ae9-7f4b-450f-a226-8d41faaa4e74 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:22.975219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885c7ae9-7f4b-450f-a226-8d41faaa4e74 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:23.017354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269a3a0-1a6d-407e-a0bc-ffccbb14d96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:23.018060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38573c96-0aaf-488f-8472-8aff254efbf4 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:23.018745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38573c96-0aaf-488f-8472-8aff254efbf4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:23.019115 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269a3a0-1a6d-407e-a0bc-ffccbb14d96a admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:23.019263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38573c96-0aaf-488f-8472-8aff254efbf4 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:23.019592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269a3a0-1a6d-407e-a0bc-ffccbb14d96a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:23.020074 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d269a3a0-1a6d-407e-a0bc-ffccbb14d96a admin admin] PATH_INFO: `/v3/projects/642bdfe7e11142a8aa9e16c2cb59be68/users/97b69b5a059846a9b1b05e33874f036d/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:23.022505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 837/1677] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:27:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:27:23.033970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146dbd0-f167-40ac-ae9a-72700bbb78a8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:23.034260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146dbd0-f167-40ac-ae9a-72700bbb78a8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:23.034624 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c146dbd0-f167-40ac-ae9a-72700bbb78a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:23.082587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:23.097523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269a3a0-1a6d-407e-a0bc-ffccbb14d96a admin admin] RBAC: Authorizing `identity:create_grant(project_id=642bdfe7e11142a8aa9e16c2cb59be68, user_id=97b69b5a059846a9b1b05e33874f036d, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:23.121702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d269a3a0-1a6d-407e-a0bc-ffccbb14d96a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:23.123887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c146dbd0-f167-40ac-ae9a-72700bbb78a8 None None] MFA Rules not processed for user `2e97119806774c2482a9886b11ab6f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:23.126594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c146dbd0-f167-40ac-ae9a-72700bbb78a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:23.191492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 841/1678] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:22 2022] PUT /identity/v3/projects/642bdfe7e11142a8aa9e16c2cb59be68/users/97b69b5a059846a9b1b05e33874f036d/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:23.197705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269a3a0-1a6d-407e-a0bc-ffccbb14d96a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:23.396042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 838/1679] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:27:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:27:23.426304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7bb06-102a-4ca4-a8b2-b30cb399a196 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:23.426930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7bb06-102a-4ca4-a8b2-b30cb399a196 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:23.428631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d7bb06-102a-4ca4-a8b2-b30cb399a196 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:23.440673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 839/1680] 10.222.0.23 () {54 vars in 960 bytes} [Tue Jul 12 13:27:23 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:27:23.447948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206cfb6-c9bd-40d8-8767-0cd369dc3cfe None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:23.448959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206cfb6-c9bd-40d8-8767-0cd369dc3cfe None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:23.449694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206cfb6-c9bd-40d8-8767-0cd369dc3cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:23.458962 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d269a3a0-1a6d-407e-a0bc-ffccbb14d96a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:23.468173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9e04-7935-4d6c-ae04-e80d578789a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:23.469663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9e04-7935-4d6c-ae04-e80d578789a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:23.470353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9e04-7935-4d6c-ae04-e80d578789a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:23.471705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e9e04-7935-4d6c-ae04-e80d578789a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:23.473903 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e9e04-7935-4d6c-ae04-e80d578789a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:23.477312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e9e04-7935-4d6c-ae04-e80d578789a7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:23.507484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c206cfb6-c9bd-40d8-8767-0cd369dc3cfe None None] MFA Rules not processed for user `2e97119806774c2482a9886b11ab6f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:23.510150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c206cfb6-c9bd-40d8-8767-0cd369dc3cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:23.517214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 842/1681] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:23 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:23.524700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9e04-7935-4d6c-ae04-e80d578789a7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:23.545619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e9e04-7935-4d6c-ae04-e80d578789a7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:23.560618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb32879-1cf4-4aed-b727-d7b953c24c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:23.562542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb32879-1cf4-4aed-b727-d7b953c24c8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:23.563039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb32879-1cf4-4aed-b727-d7b953c24c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:23.563393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb32879-1cf4-4aed-b727-d7b953c24c8f admin admin] PATH_INFO: `/v3/projects/642bdfe7e11142a8aa9e16c2cb59be68/users/97b69b5a059846a9b1b05e33874f036d/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:23.568414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb32879-1cf4-4aed-b727-d7b953c24c8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=642bdfe7e11142a8aa9e16c2cb59be68, user_id=97b69b5a059846a9b1b05e33874f036d, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:23.573793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb32879-1cf4-4aed-b727-d7b953c24c8f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:23.603548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 843/1682] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:23 2022] PUT /identity/v3/projects/642bdfe7e11142a8aa9e16c2cb59be68/users/97b69b5a059846a9b1b05e33874f036d/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:23.625320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532312d8-7244-44e1-ac37-c04c2be49c07 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:23.626661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532312d8-7244-44e1-ac37-c04c2be49c07 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:23.629127 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532312d8-7244-44e1-ac37-c04c2be49c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:23.677853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 840/1683] 10.222.0.23 () {58 vars in 1057 bytes} [Tue Jul 12 13:27:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:23.679839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206cfb6-c9bd-40d8-8767-0cd369dc3cfe None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:23.703799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206cfb6-c9bd-40d8-8767-0cd369dc3cfe None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:23.728612 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-532312d8-7244-44e1-ac37-c04c2be49c07 None None] MFA Rules not processed for user `97b69b5a059846a9b1b05e33874f036d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:23.736375 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-532312d8-7244-44e1-ac37-c04c2be49c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:23.837200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16d46c-33c0-4b9c-8013-a0a59771a29d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e97119806774c2482a9886b11ab6f70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:23.839278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16d46c-33c0-4b9c-8013-a0a59771a29d service placement] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:23.839712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16d46c-33c0-4b9c-8013-a0a59771a29d service placement] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:23.840143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c16d46c-33c0-4b9c-8013-a0a59771a29d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:23.860537 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16d46c-33c0-4b9c-8013-a0a59771a29d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:23.863267 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c16d46c-33c0-4b9c-8013-a0a59771a29d service placement] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:23.879298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 841/1684] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:23.892243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 844/1685] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:23 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:23.909784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16d46c-33c0-4b9c-8013-a0a59771a29d service placement] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:23.913056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2c16d46c-33c0-4b9c-8013-a0a59771a29d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:23.995385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:24.042955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c16d46c-33c0-4b9c-8013-a0a59771a29d service placement] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:24.047510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a950fcf-67a6-4a70-aba3-d8cae31851e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:24.048398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a950fcf-67a6-4a70-aba3-d8cae31851e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:24.048688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a950fcf-67a6-4a70-aba3-d8cae31851e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:24.049660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a950fcf-67a6-4a70-aba3-d8cae31851e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:24.064557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a950fcf-67a6-4a70-aba3-d8cae31851e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:24.067576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a950fcf-67a6-4a70-aba3-d8cae31851e2 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:24.085407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 842/1686] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:23 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:24.341123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532312d8-7244-44e1-ac37-c04c2be49c07 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:24.342836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-532312d8-7244-44e1-ac37-c04c2be49c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:24.429566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:24.472216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532312d8-7244-44e1-ac37-c04c2be49c07 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:24.478639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:24.479953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:24.480474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:24.481203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] PATH_INFO: `/v3/users/c760993f65ba499c967de8cdb207035a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:24.482908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c760993f65ba499c967de8cdb207035a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:24.487281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:24.570546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c760993f65ba499c967de8cdb207035a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:24.581474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c760993f65ba499c967de8cdb207035a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:24.582609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c760993f65ba499c967de8cdb207035a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:24.634784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c760993f65ba499c967de8cdb207035a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:24.668253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-bca9c4bb-3d1b-48a2-bdf6-e3bdec74d147 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c760993f65ba499c967de8cdb207035a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:24.693957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 845/1687] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:24 2022] DELETE /identity/v3/users/c760993f65ba499c967de8cdb207035a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:24.842727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a950fcf-67a6-4a70-aba3-d8cae31851e2 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:24.844446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5a950fcf-67a6-4a70-aba3-d8cae31851e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:24.871808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a950fcf-67a6-4a70-aba3-d8cae31851e2 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:25.020180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:25.021824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:25.024786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:25.028228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] PATH_INFO: `/v3/projects/28f68c2a85b34694a19e6a00590f3ec7` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:25.033858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28f68c2a85b34694a19e6a00590f3ec7)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:25.039019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:25.168742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28f68c2a85b34694a19e6a00590f3ec7', 'request_id': 'req-87a1f495-70e3-4921-b467-2f7e04573791'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:25.179223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28f68c2a85b34694a19e6a00590f3ec7', 'request_id': 'req-87a1f495-70e3-4921-b467-2f7e04573791'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:25.191649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28f68c2a85b34694a19e6a00590f3ec7', 'request_id': 'req-87a1f495-70e3-4921-b467-2f7e04573791'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:25.220539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] The token cache is being invalidate because project 28f68c2a85b34694a19e6a00590f3ec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:25.221409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:25.223653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-87a1f495-70e3-4921-b467-2f7e04573791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:25.231434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 843/1688] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:24 2022] DELETE /identity/v3/projects/28f68c2a85b34694a19e6a00590f3ec7 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:25.233540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20de77-f909-4ce7-8a92-5482d5b33f0d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:25.233858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20de77-f909-4ce7-8a92-5482d5b33f0d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:25.234202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a20de77-f909-4ce7-8a92-5482d5b33f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:25.290035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9a20de77-f909-4ce7-8a92-5482d5b33f0d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:25.291356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9a20de77-f909-4ce7-8a92-5482d5b33f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:25.475850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 846/1689] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:25.476815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777823d3-927e-4d69-b510-c494b3b16c12 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:25.477108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777823d3-927e-4d69-b510-c494b3b16c12 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:25.477448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777823d3-927e-4d69-b510-c494b3b16c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:25.526123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-777823d3-927e-4d69-b510-c494b3b16c12 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:25.529714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-777823d3-927e-4d69-b510-c494b3b16c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:25.550953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 844/1690] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:25.555064 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20de77-f909-4ce7-8a92-5482d5b33f0d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:25.577734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a20de77-f909-4ce7-8a92-5482d5b33f0d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:25.587449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:25.589225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:25.589789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:25.590278 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:25.591347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:25.591850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:25.595366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:25.623043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 847/1691] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:25.628445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777823d3-927e-4d69-b510-c494b3b16c12 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:25.643155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777823d3-927e-4d69-b510-c494b3b16c12 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:25.648394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cf0ca-27ac-4bcc-b00a-02b408413ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:25.649116 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cf0ca-27ac-4bcc-b00a-02b408413ae8 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:25.649427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cf0ca-27ac-4bcc-b00a-02b408413ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:25.649714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2cf0ca-27ac-4bcc-b00a-02b408413ae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:25.650440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cf0ca-27ac-4bcc-b00a-02b408413ae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:25.652890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2cf0ca-27ac-4bcc-b00a-02b408413ae8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:25.677680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:25.680276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:25.715025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815ba32-768b-4435-a2ef-ac87d4fc8547 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:25.719398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 845/1692] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:25 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:25.729986 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cf0ca-27ac-4bcc-b00a-02b408413ae8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:25.876249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:25.904892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:25.906011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:25.907016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:25.907016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] PATH_INFO: `/v3/users/7d6a4e54fe0f489f8e581062840b5077` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:25.913944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d6a4e54fe0f489f8e581062840b5077)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:25.916212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:25.923296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:26.044285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d6a4e54fe0f489f8e581062840b5077'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.060191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d6a4e54fe0f489f8e581062840b5077'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.062772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d6a4e54fe0f489f8e581062840b5077'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.076222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2cf0ca-27ac-4bcc-b00a-02b408413ae8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:26.093653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648549e4-95e2-4ff9-9b41-17ebc8bbd0c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:26.097341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648549e4-95e2-4ff9-9b41-17ebc8bbd0c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:26.098424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648549e4-95e2-4ff9-9b41-17ebc8bbd0c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:26.099075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648549e4-95e2-4ff9-9b41-17ebc8bbd0c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:26.100900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d6a4e54fe0f489f8e581062840b5077'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.102263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648549e4-95e2-4ff9-9b41-17ebc8bbd0c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:26.108055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648549e4-95e2-4ff9-9b41-17ebc8bbd0c9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:26.118650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6f4aab3e-1817-480b-af68-e57f6de53303 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d6a4e54fe0f489f8e581062840b5077'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.130460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 848/1693] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:25 2022] DELETE /identity/v3/users/7d6a4e54fe0f489f8e581062840b5077 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:26.134771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fcc3b-4d52-47c1-be70-fe091ca3d737 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:26.135123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fcc3b-4d52-47c1-be70-fe091ca3d737 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:26.135624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1fcc3b-4d52-47c1-be70-fe091ca3d737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:26.177668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4c1fcc3b-4d52-47c1-be70-fe091ca3d737 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:26.179257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4c1fcc3b-4d52-47c1-be70-fe091ca3d737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:26.213081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 846/1694] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:25 2022] POST /identity/v3/users => generated 293 bytes in 492 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:26.223156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648549e4-95e2-4ff9-9b41-17ebc8bbd0c9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:26.233956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-648549e4-95e2-4ff9-9b41-17ebc8bbd0c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:26.294630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648549e4-95e2-4ff9-9b41-17ebc8bbd0c9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:26.365661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 849/1695] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:27:26.390105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fcc3b-4d52-47c1-be70-fe091ca3d737 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:26.391923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4c1fcc3b-4d52-47c1-be70-fe091ca3d737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:26.445638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df91c41-5d60-4137-86c2-e12d3d0d95de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:26.446425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df91c41-5d60-4137-86c2-e12d3d0d95de admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:26.446753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df91c41-5d60-4137-86c2-e12d3d0d95de admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:26.447077 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df91c41-5d60-4137-86c2-e12d3d0d95de admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:26.447908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df91c41-5d60-4137-86c2-e12d3d0d95de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:26.449828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df91c41-5d60-4137-86c2-e12d3d0d95de admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:26.470417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 847/1696] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:26.471938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df91c41-5d60-4137-86c2-e12d3d0d95de admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:26.476468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1df91c41-5d60-4137-86c2-e12d3d0d95de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:26.500609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df91c41-5d60-4137-86c2-e12d3d0d95de admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:26.519824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:26.523487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:26.523866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:26.525215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] PATH_INFO: `/v3/projects/74c80a6b1b314802a9b61b486583d8c3` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:26.535220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74c80a6b1b314802a9b61b486583d8c3)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:26.538973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:26.561415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1fcc3b-4d52-47c1-be70-fe091ca3d737 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:26.578680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb46faa-ba53-4fde-ac79-a1f94cb9fda2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:26.579394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb46faa-ba53-4fde-ac79-a1f94cb9fda2 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:26.579707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb46faa-ba53-4fde-ac79-a1f94cb9fda2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:26.581010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb46faa-ba53-4fde-ac79-a1f94cb9fda2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:26.616576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb46faa-ba53-4fde-ac79-a1f94cb9fda2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:26.617713 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb46faa-ba53-4fde-ac79-a1f94cb9fda2 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:26.649444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 850/1697] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:26.651826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb46faa-ba53-4fde-ac79-a1f94cb9fda2 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:26.654312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74c80a6b1b314802a9b61b486583d8c3', 'request_id': 'req-8c3258a5-d05b-4a89-88f0-507a7697e1d5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.669127 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb46faa-ba53-4fde-ac79-a1f94cb9fda2 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:26.676287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74c80a6b1b314802a9b61b486583d8c3', 'request_id': 'req-8c3258a5-d05b-4a89-88f0-507a7697e1d5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.684587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:26.693038 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74c80a6b1b314802a9b61b486583d8c3', 'request_id': 'req-8c3258a5-d05b-4a89-88f0-507a7697e1d5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.724445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] The token cache is being invalidate because project 74c80a6b1b314802a9b61b486583d8c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:26.724749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.727025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:26.730494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 848/1698] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:26 2022] DELETE /identity/v3/projects/74c80a6b1b314802a9b61b486583d8c3 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:26.736106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:26.739282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:26.868656 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:26.880196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf085265-52e4-4bb6-ba9d-80356ba2076a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:26.885623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085265-52e4-4bb6-ba9d-80356ba2076a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:26.887058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085265-52e4-4bb6-ba9d-80356ba2076a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:26.887340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf085265-52e4-4bb6-ba9d-80356ba2076a admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:26.889446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf085265-52e4-4bb6-ba9d-80356ba2076a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:26.895473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf085265-52e4-4bb6-ba9d-80356ba2076a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:26.929141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 851/1699] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:26.931367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3258a5-d05b-4a89-88f0-507a7697e1d5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:26.935360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf085265-52e4-4bb6-ba9d-80356ba2076a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:26.939803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bf085265-52e4-4bb6-ba9d-80356ba2076a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:26.950439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:26.951668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:26.952019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:26.952377 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] PATH_INFO: `/v3/users/cb5f426772e747a79144cfde79919fe3` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:26.954160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb5f426772e747a79144cfde79919fe3)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:26.956236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:26.979391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf085265-52e4-4bb6-ba9d-80356ba2076a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:26.996824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a52b5c6-7838-4e64-918c-baa417bb1024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:27.002072 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52b5c6-7838-4e64-918c-baa417bb1024 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:27.003241 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52b5c6-7838-4e64-918c-baa417bb1024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:27.004256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a52b5c6-7838-4e64-918c-baa417bb1024 admin admin] PATH_INFO: `/v3/projects/a2cc9d1b356e4e8aba32a63fbd11a985/users/17cae227dd804fc3becb13315d4797dd/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:27.062717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52b5c6-7838-4e64-918c-baa417bb1024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2cc9d1b356e4e8aba32a63fbd11a985, user_id=17cae227dd804fc3becb13315d4797dd, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:27.069833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a52b5c6-7838-4e64-918c-baa417bb1024 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:27.078258 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb5f426772e747a79144cfde79919fe3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:27.094684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb5f426772e747a79144cfde79919fe3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:27.096750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb5f426772e747a79144cfde79919fe3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:27.113329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb5f426772e747a79144cfde79919fe3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:27.134228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 852/1700] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:26 2022] PUT /identity/v3/projects/a2cc9d1b356e4e8aba32a63fbd11a985/users/17cae227dd804fc3becb13315d4797dd/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:27.150808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a7b9-e5c8-457b-a3dd-f092ce7d8c40 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:27.152045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a7b9-e5c8-457b-a3dd-f092ce7d8c40 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:27.152337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2567a7b9-e5c8-457b-a3dd-f092ce7d8c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:27.172018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb5f426772e747a79144cfde79919fe3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:27.197161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 849/1701] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:26 2022] DELETE /identity/v3/users/cb5f426772e747a79144cfde79919fe3 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:27.246714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2567a7b9-e5c8-457b-a3dd-f092ce7d8c40 None None] MFA Rules not processed for user `17cae227dd804fc3becb13315d4797dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:27.254002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2567a7b9-e5c8-457b-a3dd-f092ce7d8c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:27.458679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 853/1702] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:27.464030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:27.466454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:27.703230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16974eb1-49ac-453f-86b7-c2050fed1fe0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:27.714960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c82e4-eb28-44c3-a17e-88c70499412b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:27.715813 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c82e4-eb28-44c3-a17e-88c70499412b admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:27.716347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c82e4-eb28-44c3-a17e-88c70499412b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:27.716922 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250c82e4-eb28-44c3-a17e-88c70499412b admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:27.717825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c82e4-eb28-44c3-a17e-88c70499412b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:27.724879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250c82e4-eb28-44c3-a17e-88c70499412b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:27.806178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 850/1703] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:27 2022] POST /identity/v3/projects => generated 387 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:27.810077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a7b9-e5c8-457b-a3dd-f092ce7d8c40 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:27.833550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2567a7b9-e5c8-457b-a3dd-f092ce7d8c40 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:27.987860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf76408-40cb-4c4e-9e2b-5d20b2d35492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:27.989926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf76408-40cb-4c4e-9e2b-5d20b2d35492 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:27.990393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf76408-40cb-4c4e-9e2b-5d20b2d35492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:27.992144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf76408-40cb-4c4e-9e2b-5d20b2d35492 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:27.995148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf76408-40cb-4c4e-9e2b-5d20b2d35492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:27.998545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf76408-40cb-4c4e-9e2b-5d20b2d35492 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:28.047858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 854/1704] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:27 2022] POST /identity/v3/users => generated 300 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:28.049781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c82e4-eb28-44c3-a17e-88c70499412b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:28.067900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250c82e4-eb28-44c3-a17e-88c70499412b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:28.083924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a051ec-a600-4dfe-8df4-2de5d9101f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:28.084469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a051ec-a600-4dfe-8df4-2de5d9101f44 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:28.084755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a051ec-a600-4dfe-8df4-2de5d9101f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:28.085033 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a051ec-a600-4dfe-8df4-2de5d9101f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:28.085836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a051ec-a600-4dfe-8df4-2de5d9101f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:28.087689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a051ec-a600-4dfe-8df4-2de5d9101f44 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:28.124892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf76408-40cb-4c4e-9e2b-5d20b2d35492 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:28.125327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 851/1705] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:28.132513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:28.151751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf76408-40cb-4c4e-9e2b-5d20b2d35492 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:28.167302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb2575-af2e-493d-9604-c0a426902af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:28.168428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb2575-af2e-493d-9604-c0a426902af1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:28.168752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb2575-af2e-493d-9604-c0a426902af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:28.169050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bb2575-af2e-493d-9604-c0a426902af1 admin admin] PATH_INFO: `/v3/projects/7f30a18ba5c646868927ec9cdd286614/users/3603bc39008f4c8199e13e54d8832205/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:28.211028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb2575-af2e-493d-9604-c0a426902af1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f30a18ba5c646868927ec9cdd286614, user_id=3603bc39008f4c8199e13e54d8832205, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:28.214918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bb2575-af2e-493d-9604-c0a426902af1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:28.241609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 855/1706] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:28 2022] PUT /identity/v3/projects/7f30a18ba5c646868927ec9cdd286614/users/3603bc39008f4c8199e13e54d8832205/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:28.245712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a051ec-a600-4dfe-8df4-2de5d9101f44 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:28.262053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a051ec-a600-4dfe-8df4-2de5d9101f44 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:28.342445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:28.395099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afae7f-be19-4c45-afc5-92b6d40b8a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:28.395428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afae7f-be19-4c45-afc5-92b6d40b8a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:28.395725 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afae7f-be19-4c45-afc5-92b6d40b8a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:28.396012 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9afae7f-be19-4c45-afc5-92b6d40b8a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:28.396920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afae7f-be19-4c45-afc5-92b6d40b8a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:28.398614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9afae7f-be19-4c45-afc5-92b6d40b8a75 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:28.416979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb2575-af2e-493d-9604-c0a426902af1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:28.420437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-36bb2575-af2e-493d-9604-c0a426902af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:28.435383 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 852/1707] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:28.440536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afae7f-be19-4c45-afc5-92b6d40b8a75 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:28.453831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bb2575-af2e-493d-9604-c0a426902af1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:28.478353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:28.479278 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:28.479596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:28.479856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] PATH_INFO: `/v3/projects/730610bab7284e04bb14fa0752bc3289` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:28.482049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=730610bab7284e04bb14fa0752bc3289)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:28.485859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9afae7f-be19-4c45-afc5-92b6d40b8a75 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:28.490926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:28.512916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bef99-5a80-44fb-aaec-635121feb3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:28.514336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bef99-5a80-44fb-aaec-635121feb3dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:28.514769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bef99-5a80-44fb-aaec-635121feb3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:28.515283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239bef99-5a80-44fb-aaec-635121feb3dc admin admin] PATH_INFO: `/v3/projects/7f30a18ba5c646868927ec9cdd286614/users/3603bc39008f4c8199e13e54d8832205/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:28.520580 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bef99-5a80-44fb-aaec-635121feb3dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f30a18ba5c646868927ec9cdd286614, user_id=3603bc39008f4c8199e13e54d8832205, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:28.523748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239bef99-5a80-44fb-aaec-635121feb3dc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:28.557978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 853/1708] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:28 2022] PUT /identity/v3/projects/7f30a18ba5c646868927ec9cdd286614/users/3603bc39008f4c8199e13e54d8832205/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:28.559480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bef99-5a80-44fb-aaec-635121feb3dc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:28.571097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-239bef99-5a80-44fb-aaec-635121feb3dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:28.618364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '730610bab7284e04bb14fa0752bc3289', 'request_id': 'req-995bb85b-eb16-45c5-8fc8-ed277537aa11'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:28.634366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '730610bab7284e04bb14fa0752bc3289', 'request_id': 'req-995bb85b-eb16-45c5-8fc8-ed277537aa11'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:28.655263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '730610bab7284e04bb14fa0752bc3289', 'request_id': 'req-995bb85b-eb16-45c5-8fc8-ed277537aa11'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:28.683139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] The token cache is being invalidate because project 730610bab7284e04bb14fa0752bc3289 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:28.683578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:28.685601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-995bb85b-eb16-45c5-8fc8-ed277537aa11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:28.687474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 856/1709] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:28 2022] DELETE /identity/v3/projects/730610bab7284e04bb14fa0752bc3289 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:28.690310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0300680-792a-4729-ad21-feb8f0ec2735 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:28.690674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0300680-792a-4729-ad21-feb8f0ec2735 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:28.691184 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0300680-792a-4729-ad21-feb8f0ec2735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:28.739246 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239bef99-5a80-44fb-aaec-635121feb3dc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:28.772110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b0300680-792a-4729-ad21-feb8f0ec2735 None None] MFA Rules not processed for user `3603bc39008f4c8199e13e54d8832205`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:28.779159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b0300680-792a-4729-ad21-feb8f0ec2735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:28.863104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:28.864120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:28.864417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:28.864700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] PATH_INFO: `/v3/projects/3e93280502854638ad841ef17953ad6c` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:28.866049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e93280502854638ad841ef17953ad6c)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:28.868755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:28.948907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e93280502854638ad841ef17953ad6c', 'request_id': 'req-abcd609e-cf55-44ea-b70e-b8dd4511a650'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:28.956764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 857/1710] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:28 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:28.957953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0300680-792a-4729-ad21-feb8f0ec2735 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:28.958082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e93280502854638ad841ef17953ad6c', 'request_id': 'req-abcd609e-cf55-44ea-b70e-b8dd4511a650'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:28.962429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b0300680-792a-4729-ad21-feb8f0ec2735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:28.968404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e93280502854638ad841ef17953ad6c', 'request_id': 'req-abcd609e-cf55-44ea-b70e-b8dd4511a650'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:28.987878 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:28.991890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0300680-792a-4729-ad21-feb8f0ec2735 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:29.008481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] The token cache is being invalidate because project 3e93280502854638ad841ef17953ad6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:29.009052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:29.010322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-abcd609e-cf55-44ea-b70e-b8dd4511a650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:29.014082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 854/1711] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:28 2022] DELETE /identity/v3/projects/3e93280502854638ad841ef17953ad6c => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:29.017273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218c63f-d9c1-43ed-833d-863e1c9ad0ab None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:29.018072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218c63f-d9c1-43ed-833d-863e1c9ad0ab None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:29.019738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218c63f-d9c1-43ed-833d-863e1c9ad0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:29.067783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:29.075320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e218c63f-d9c1-43ed-833d-863e1c9ad0ab None None] MFA Rules not processed for user `f8de4128718d4837af77d427973d30be`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:29.077692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e218c63f-d9c1-43ed-833d-863e1c9ad0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:29.087003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:29.087910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:29.088199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:29.088477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] PATH_INFO: `/v3/users/97b69b5a059846a9b1b05e33874f036d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:29.097535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] RBAC: Authorizing `identity:delete_user(user_id=97b69b5a059846a9b1b05e33874f036d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:29.097535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:29.178082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97b69b5a059846a9b1b05e33874f036d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:29.214148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97b69b5a059846a9b1b05e33874f036d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:29.215411 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97b69b5a059846a9b1b05e33874f036d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:29.228687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 855/1712] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:29.228687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218c63f-d9c1-43ed-833d-863e1c9ad0ab None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:29.228687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e218c63f-d9c1-43ed-833d-863e1c9ad0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:29.254272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97b69b5a059846a9b1b05e33874f036d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:29.265654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-30448bba-0d05-482d-907d-4be94caf647c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97b69b5a059846a9b1b05e33874f036d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:29.296083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 858/1713] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:28 2022] DELETE /identity/v3/users/97b69b5a059846a9b1b05e33874f036d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:29.300438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69451910-b5ba-47cb-85ed-1d83ada6964e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:29.303248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69451910-b5ba-47cb-85ed-1d83ada6964e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:29.306151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69451910-b5ba-47cb-85ed-1d83ada6964e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:29.354310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-69451910-b5ba-47cb-85ed-1d83ada6964e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:29.357025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-69451910-b5ba-47cb-85ed-1d83ada6964e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:29.443822 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218c63f-d9c1-43ed-833d-863e1c9ad0ab None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:29.612012 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 859/1714] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:29.613845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208cdc-7c30-45a7-bd4a-af2d40388e13 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:29.615469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208cdc-7c30-45a7-bd4a-af2d40388e13 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:29.617076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74208cdc-7c30-45a7-bd4a-af2d40388e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:29.680074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43b615b-052e-4dc5-88a1-5013e0a5e00d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:29.681234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b615b-052e-4dc5-88a1-5013e0a5e00d service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:29.681234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b615b-052e-4dc5-88a1-5013e0a5e00d service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:29.681447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43b615b-052e-4dc5-88a1-5013e0a5e00d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:29.686882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e43b615b-052e-4dc5-88a1-5013e0a5e00d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:29.706785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-74208cdc-7c30-45a7-bd4a-af2d40388e13 None None] MFA Rules not processed for user `c4b710bbdf0c4d58b4f3ee7ef7410c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:29.711681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-74208cdc-7c30-45a7-bd4a-af2d40388e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:29.848185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b615b-052e-4dc5-88a1-5013e0a5e00d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:29.851995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43b615b-052e-4dc5-88a1-5013e0a5e00d service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:29.883178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 856/1715] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 658 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:29.884749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151051f1-83ae-4c7c-80c1-f9fb4ed326b0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:29.885046 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151051f1-83ae-4c7c-80c1-f9fb4ed326b0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:29.885330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151051f1-83ae-4c7c-80c1-f9fb4ed326b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:29.948696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:29.969263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-151051f1-83ae-4c7c-80c1-f9fb4ed326b0 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:29.972131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-151051f1-83ae-4c7c-80c1-f9fb4ed326b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:29.990205 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:30.008201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 857/1716] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:30.014321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151051f1-83ae-4c7c-80c1-f9fb4ed326b0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:30.031506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 860/1717] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:30.036502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74208cdc-7c30-45a7-bd4a-af2d40388e13 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:30.038392 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151051f1-83ae-4c7c-80c1-f9fb4ed326b0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:30.045803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-74208cdc-7c30-45a7-bd4a-af2d40388e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:30.053464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:30.054157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:30.054453 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:30.054734 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:30.055495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:30.055798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:30.058271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:30.074090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74208cdc-7c30-45a7-bd4a-af2d40388e13 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:30.085231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 858/1718] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:30.097205 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:30.097599 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:30.098169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:30.098449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:30.098725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] PATH_INFO: `/v3/users/17cae227dd804fc3becb13315d4797dd` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:30.104654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] RBAC: Authorizing `identity:delete_user(user_id=17cae227dd804fc3becb13315d4797dd)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:30.106555 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:30.119540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:30.256699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17cae227dd804fc3becb13315d4797dd'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:30.294358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17cae227dd804fc3becb13315d4797dd'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:30.296446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17cae227dd804fc3becb13315d4797dd'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:30.308097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7631717-99a6-41e7-83a0-2f2b8cf09f00 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:30.321614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03810a21-deab-4d0e-8f27-a0f55495f76d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:30.322522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03810a21-deab-4d0e-8f27-a0f55495f76d service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:30.322672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03810a21-deab-4d0e-8f27-a0f55495f76d service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:30.322844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03810a21-deab-4d0e-8f27-a0f55495f76d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:30.336617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17cae227dd804fc3becb13315d4797dd'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:30.340461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03810a21-deab-4d0e-8f27-a0f55495f76d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:30.342636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03810a21-deab-4d0e-8f27-a0f55495f76d service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:30.355719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17cae227dd804fc3becb13315d4797dd'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:30.375680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 861/1719] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:30 2022] DELETE /identity/v3/users/17cae227dd804fc3becb13315d4797dd => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:30.375926 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 859/1720] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:30.377036 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:30.378215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7de5cb-563f-4b9b-97c2-13165f2776ac None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:30.378519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7de5cb-563f-4b9b-97c2-13165f2776ac None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:30.378810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7de5cb-563f-4b9b-97c2-13165f2776ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:30.382899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:30.418592 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fd7de5cb-563f-4b9b-97c2-13165f2776ac None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:30.419869 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fd7de5cb-563f-4b9b-97c2-13165f2776ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:30.558960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 860/1721] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:30.613654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd08d74-5946-4033-9d63-b80b82f9362a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:30.622677 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7de5cb-563f-4b9b-97c2-13165f2776ac None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:30.624389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fd7de5cb-563f-4b9b-97c2-13165f2776ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:30.628195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297201bf-6cf7-4524-b696-2399819958f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:30.629183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297201bf-6cf7-4524-b696-2399819958f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:30.629658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297201bf-6cf7-4524-b696-2399819958f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:30.630788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297201bf-6cf7-4524-b696-2399819958f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:30.632896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297201bf-6cf7-4524-b696-2399819958f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:30.637470 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297201bf-6cf7-4524-b696-2399819958f2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:30.714224 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 862/1722] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:30 2022] POST /identity/v3/projects => generated 375 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:30.718317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297201bf-6cf7-4524-b696-2399819958f2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:30.845403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7de5cb-563f-4b9b-97c2-13165f2776ac None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:30.870848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:30.907150 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297201bf-6cf7-4524-b696-2399819958f2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:30.925448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:30.931218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7461647-1db4-4887-85b7-4500a76dfd97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:30.932271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7461647-1db4-4887-85b7-4500a76dfd97 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:30.932570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7461647-1db4-4887-85b7-4500a76dfd97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:30.932861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7461647-1db4-4887-85b7-4500a76dfd97 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:30.933674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7461647-1db4-4887-85b7-4500a76dfd97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:30.935700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7461647-1db4-4887-85b7-4500a76dfd97 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:30.968114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6759809-c316-4431-81e0-5fc6cfb4a6a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:30.970043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6759809-c316-4431-81e0-5fc6cfb4a6a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:30.973971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6759809-c316-4431-81e0-5fc6cfb4a6a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:30.974577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6759809-c316-4431-81e0-5fc6cfb4a6a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.000067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6759809-c316-4431-81e0-5fc6cfb4a6a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:31.000067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6759809-c316-4431-81e0-5fc6cfb4a6a8 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:31.031050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 861/1723] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:27:31.032086 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6759809-c316-4431-81e0-5fc6cfb4a6a8 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:31.037440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 863/1724] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:30 2022] POST /identity/v3/users => generated 288 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:31.039084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7461647-1db4-4887-85b7-4500a76dfd97 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:31.041164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f7461647-1db4-4887-85b7-4500a76dfd97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:31.047639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a6759809-c316-4431-81e0-5fc6cfb4a6a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:31.083052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6759809-c316-4431-81e0-5fc6cfb4a6a8 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:31.084680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7461647-1db4-4887-85b7-4500a76dfd97 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:31.093985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:31.095127 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:31.095445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:31.098998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] PATH_INFO: `/v3/users/0329c5da1b12499991e966170aeabbd9` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.101613 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0329c5da1b12499991e966170aeabbd9)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:31.102510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:31.103581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:31.103873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:31.104361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:31.104851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] PATH_INFO: `/v3/projects/642bdfe7e11142a8aa9e16c2cb59be68` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.107596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] RBAC: Authorizing `identity:delete_project(project_id=642bdfe7e11142a8aa9e16c2cb59be68)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:31.110106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:31.211431 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '642bdfe7e11142a8aa9e16c2cb59be68', 'request_id': 'req-9ed2e813-9c6b-4782-9977-861cebd42cce'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.224348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '642bdfe7e11142a8aa9e16c2cb59be68', 'request_id': 'req-9ed2e813-9c6b-4782-9977-861cebd42cce'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.260627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '642bdfe7e11142a8aa9e16c2cb59be68', 'request_id': 'req-9ed2e813-9c6b-4782-9977-861cebd42cce'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.262442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0329c5da1b12499991e966170aeabbd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.275099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0329c5da1b12499991e966170aeabbd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.276024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0329c5da1b12499991e966170aeabbd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.296323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] The token cache is being invalidate because project 642bdfe7e11142a8aa9e16c2cb59be68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:31.297085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.298235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.299222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0329c5da1b12499991e966170aeabbd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.317451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 862/1725] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:31 2022] DELETE /identity/v3/projects/642bdfe7e11142a8aa9e16c2cb59be68 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:31.318476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:31.320779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:31.328452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-26840395-abbe-49f8-85ee-8b742c80f568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0329c5da1b12499991e966170aeabbd9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:31.339735 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 864/1726] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:31 2022] DELETE /identity/v3/users/0329c5da1b12499991e966170aeabbd9 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:31.345630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3ef6c-5c63-460e-ad9d-a65d984df20f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:31.346040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3ef6c-5c63-460e-ad9d-a65d984df20f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:31.346227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3ef6c-5c63-460e-ad9d-a65d984df20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.389187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-02f3ef6c-5c63-460e-ad9d-a65d984df20f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:31.393229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-02f3ef6c-5c63-460e-ad9d-a65d984df20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:31.497673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2e813-9c6b-4782-9977-861cebd42cce admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:31.546694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 865/1727] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:31.556204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741e555-be86-4b76-aa1e-31c655367fc6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:31.556496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741e555-be86-4b76-aa1e-31c655367fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:31.562928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741e555-be86-4b76-aa1e-31c655367fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.587168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5741e555-be86-4b76-aa1e-31c655367fc6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:31.588409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5741e555-be86-4b76-aa1e-31c655367fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:31.600919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 866/1728] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:31.602506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff485e8-38e2-4522-8774-4b4c617a2535 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:31.602799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff485e8-38e2-4522-8774-4b4c617a2535 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:31.603079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff485e8-38e2-4522-8774-4b4c617a2535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.645821 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7ff485e8-38e2-4522-8774-4b4c617a2535 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:31.647443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7ff485e8-38e2-4522-8774-4b4c617a2535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:31.665002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 867/1729] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:31.666322 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff485e8-38e2-4522-8774-4b4c617a2535 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:31.688989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff485e8-38e2-4522-8774-4b4c617a2535 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:31.708493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:31.709758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:31.710247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:31.710806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.711826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:31.712351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:31.714781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:31.735045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 868/1730] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:31.737362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:31.753866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d92e64-0168-4cf2-ac1c-a24e605ca1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:31.754558 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:31.756217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d92e64-0168-4cf2-ac1c-a24e605ca1d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:31.756508 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d92e64-0168-4cf2-ac1c-a24e605ca1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:31.756794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d92e64-0168-4cf2-ac1c-a24e605ca1d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.758714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d92e64-0168-4cf2-ac1c-a24e605ca1d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:31.765931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d92e64-0168-4cf2-ac1c-a24e605ca1d8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:31.773509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:31.799382 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 863/1731] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 481 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:31.801231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d92e64-0168-4cf2-ac1c-a24e605ca1d8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:31.831946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d92e64-0168-4cf2-ac1c-a24e605ca1d8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:31.840368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835719e-0d3e-4019-ac7e-7cee46072c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:31.841404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835719e-0d3e-4019-ac7e-7cee46072c83 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:31.841797 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835719e-0d3e-4019-ac7e-7cee46072c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:31.842060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835719e-0d3e-4019-ac7e-7cee46072c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.842404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:31.843758 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835719e-0d3e-4019-ac7e-7cee46072c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:31.848026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835719e-0d3e-4019-ac7e-7cee46072c83 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:31.914905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 864/1732] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:31 2022] POST /identity/v3/projects => generated 373 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:31.914905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6916b-6b3f-4e16-9a7b-86512923fd7e None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:31.914905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6916b-6b3f-4e16-9a7b-86512923fd7e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:31.921206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6916b-6b3f-4e16-9a7b-86512923fd7e None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:31.926603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 865/1733] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jul 12 13:27:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:31.928724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6916b-6b3f-4e16-9a7b-86512923fd7e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:31.946840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401d9edc-4be5-4c42-8d41-b27ef1865319 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:31.954111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b6e6916b-6b3f-4e16-9a7b-86512923fd7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:32.078815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44eb84-9e31-4c3e-a6bf-097cfec03f8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:32.079652 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44eb84-9e31-4c3e-a6bf-097cfec03f8d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.080599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44eb84-9e31-4c3e-a6bf-097cfec03f8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.082321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44eb84-9e31-4c3e-a6bf-097cfec03f8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.102260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44eb84-9e31-4c3e-a6bf-097cfec03f8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:32.103928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6916b-6b3f-4e16-9a7b-86512923fd7e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:32.104169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c44eb84-9e31-4c3e-a6bf-097cfec03f8d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:32.115644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad403447-2774-4b84-a63e-c41188146b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:32.116798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad403447-2774-4b84-a63e-c41188146b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.117507 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad403447-2774-4b84-a63e-c41188146b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.117944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad403447-2774-4b84-a63e-c41188146b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.119019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad403447-2774-4b84-a63e-c41188146b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:32.121072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad403447-2774-4b84-a63e-c41188146b29 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:32.141026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 866/1734] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:31 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:32.141300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 869/1735] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:32.142484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad403447-2774-4b84-a63e-c41188146b29 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:32.144260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319384-c718-45dd-89e0-e00067afa20d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.145405 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319384-c718-45dd-89e0-e00067afa20d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.145784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f319384-c718-45dd-89e0-e00067afa20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.158821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad403447-2774-4b84-a63e-c41188146b29 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:32.178132 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3bc4-4676-4568-910d-a2cbe5103078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:32.179193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3bc4-4676-4568-910d-a2cbe5103078 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.179682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3bc4-4676-4568-910d-a2cbe5103078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.180135 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff3bc4-4676-4568-910d-a2cbe5103078 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.181192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff3bc4-4676-4568-910d-a2cbe5103078 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:32.184641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff3bc4-4676-4568-910d-a2cbe5103078 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:32.225911 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8f319384-c718-45dd-89e0-e00067afa20d None None] MFA Rules not processed for user `83837e4f8f9b4e538658876cfb56db16`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:32.227473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8f319384-c718-45dd-89e0-e00067afa20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:32.230098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 867/1736] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:32 2022] POST /identity/v3/users => generated 287 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:32.231160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3bc4-4676-4568-910d-a2cbe5103078 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:32.243435 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 870/1737] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:27:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:32.244419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f319384-c718-45dd-89e0-e00067afa20d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:32.259369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff3bc4-4676-4568-910d-a2cbe5103078 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:32.265749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9300a-8421-4897-bafb-bab0116ad82f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:32.267311 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9300a-8421-4897-bafb-bab0116ad82f admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.267999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9300a-8421-4897-bafb-bab0116ad82f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.268619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f9300a-8421-4897-bafb-bab0116ad82f admin admin] PATH_INFO: `/v3/projects/d0dbb756b60f4ad9a4dcf930a4c1c090/users/34ee97f7c30d44749720076b02882892/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.282157 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f319384-c718-45dd-89e0-e00067afa20d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:32.290209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f164a-8ff2-41ae-b979-bb4a91ce0992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:32.291483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f164a-8ff2-41ae-b979-bb4a91ce0992 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.291801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f164a-8ff2-41ae-b979-bb4a91ce0992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.292087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297f164a-8ff2-41ae-b979-bb4a91ce0992 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.293072 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f164a-8ff2-41ae-b979-bb4a91ce0992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:32.295658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297f164a-8ff2-41ae-b979-bb4a91ce0992 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:32.313628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 871/1738] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:32.314855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f164a-8ff2-41ae-b979-bb4a91ce0992 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:32.319382 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9300a-8421-4897-bafb-bab0116ad82f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0dbb756b60f4ad9a4dcf930a4c1c090, user_id=34ee97f7c30d44749720076b02882892, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:32.327618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f9300a-8421-4897-bafb-bab0116ad82f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:32.342116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297f164a-8ff2-41ae-b979-bb4a91ce0992 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:32.349771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c481d44-3edc-49e2-81bc-76521b857efa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:32.350350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c481d44-3edc-49e2-81bc-76521b857efa service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.351351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c481d44-3edc-49e2-81bc-76521b857efa service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.351584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c481d44-3edc-49e2-81bc-76521b857efa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.363647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 868/1739] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:32 2022] PUT /identity/v3/projects/d0dbb756b60f4ad9a4dcf930a4c1c090/users/34ee97f7c30d44749720076b02882892/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:32.377714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9300a-8421-4897-bafb-bab0116ad82f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:32.393225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c481d44-3edc-49e2-81bc-76521b857efa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:32.395077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c481d44-3edc-49e2-81bc-76521b857efa service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:32.427541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 872/1740] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:32.428988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd4c97-89b8-4e57-abb8-62db55c562b8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.429288 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd4c97-89b8-4e57-abb8-62db55c562b8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.429616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd4c97-89b8-4e57-abb8-62db55c562b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.474756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f9300a-8421-4897-bafb-bab0116ad82f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:32.487892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302e892-54f5-40f7-8f56-0b63e9ef3c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:32.488692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302e892-54f5-40f7-8f56-0b63e9ef3c09 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.489053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302e892-54f5-40f7-8f56-0b63e9ef3c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.489429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302e892-54f5-40f7-8f56-0b63e9ef3c09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.490284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302e892-54f5-40f7-8f56-0b63e9ef3c09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:32.500384 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302e892-54f5-40f7-8f56-0b63e9ef3c09 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:32.515759 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 869/1741] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:32.516642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-73fd4c97-89b8-4e57-abb8-62db55c562b8 None None] MFA Rules not processed for user `34ee97f7c30d44749720076b02882892`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:32.521797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-73fd4c97-89b8-4e57-abb8-62db55c562b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:32.526421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302e892-54f5-40f7-8f56-0b63e9ef3c09 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:32.550151 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302e892-54f5-40f7-8f56-0b63e9ef3c09 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:32.559370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e681bc-b1a9-4b98-9f9b-d3c4f2acb57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:32.560621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e681bc-b1a9-4b98-9f9b-d3c4f2acb57f admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.561084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e681bc-b1a9-4b98-9f9b-d3c4f2acb57f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.567791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e681bc-b1a9-4b98-9f9b-d3c4f2acb57f admin admin] PATH_INFO: `/v3/projects/dd4b3ed2049c42ea97e262b5280f0f5a/users/4863b2cc79584ba5932770056a47f81c/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.590757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e681bc-b1a9-4b98-9f9b-d3c4f2acb57f admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd4b3ed2049c42ea97e262b5280f0f5a, user_id=4863b2cc79584ba5932770056a47f81c, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:32.596467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e681bc-b1a9-4b98-9f9b-d3c4f2acb57f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:32.641217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 870/1742] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:32 2022] PUT /identity/v3/projects/dd4b3ed2049c42ea97e262b5280f0f5a/users/4863b2cc79584ba5932770056a47f81c/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:32.645314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6ce7-d2c6-4dfd-910d-91dae4e521b4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:32.645736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6ce7-d2c6-4dfd-910d-91dae4e521b4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:32.646199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a6ce7-d2c6-4dfd-910d-91dae4e521b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:32.667246 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:32.773722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 873/1743] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:32.780777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0f1a6ce7-d2c6-4dfd-910d-91dae4e521b4 None None] MFA Rules not processed for user `4863b2cc79584ba5932770056a47f81c`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:32.784774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0f1a6ce7-d2c6-4dfd-910d-91dae4e521b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:32.788430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd4c97-89b8-4e57-abb8-62db55c562b8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:32.813575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd4c97-89b8-4e57-abb8-62db55c562b8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:32.938835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:33.025769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571134f0-36fa-4284-aa98-a6c131d3c557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:33.032364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571134f0-36fa-4284-aa98-a6c131d3c557 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:33.032767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571134f0-36fa-4284-aa98-a6c131d3c557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:33.034054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571134f0-36fa-4284-aa98-a6c131d3c557 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:33.035056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 871/1744] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:32 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:33.036016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a6ce7-d2c6-4dfd-910d-91dae4e521b4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:33.038550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571134f0-36fa-4284-aa98-a6c131d3c557 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:33.040691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571134f0-36fa-4284-aa98-a6c131d3c557 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:33.090945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a6ce7-d2c6-4dfd-910d-91dae4e521b4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:33.107250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e1e6f-9a9c-46b0-8a44-600300e10e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:33.108360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e1e6f-9a9c-46b0-8a44-600300e10e43 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:33.115368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e1e6f-9a9c-46b0-8a44-600300e10e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:33.115982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1e1e6f-9a9c-46b0-8a44-600300e10e43 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:33.120266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e1e6f-9a9c-46b0-8a44-600300e10e43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:33.123891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1e1e6f-9a9c-46b0-8a44-600300e10e43 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:33.147430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 874/1745] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:32 2022] POST /identity/v3/projects => generated 375 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:27:33.152915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571134f0-36fa-4284-aa98-a6c131d3c557 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:33.204153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 872/1746] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:33 2022] POST /identity/v3/projects => generated 371 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:33.212303 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e1e6f-9a9c-46b0-8a44-600300e10e43 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:33.317588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571134f0-36fa-4284-aa98-a6c131d3c557 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:33.477283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ec3b7-5fd4-4c2c-8ee0-4787d1ddba32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:33.478421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ec3b7-5fd4-4c2c-8ee0-4787d1ddba32 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:33.478421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ec3b7-5fd4-4c2c-8ee0-4787d1ddba32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:33.478677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83ec3b7-5fd4-4c2c-8ee0-4787d1ddba32 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:33.479293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ec3b7-5fd4-4c2c-8ee0-4787d1ddba32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:33.487691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83ec3b7-5fd4-4c2c-8ee0-4787d1ddba32 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:33.513488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1e1e6f-9a9c-46b0-8a44-600300e10e43 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:33.538509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db40114-9d42-4248-9f0b-df7c6d74616d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:33.543316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40114-9d42-4248-9f0b-df7c6d74616d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:33.545047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40114-9d42-4248-9f0b-df7c6d74616d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:33.545752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db40114-9d42-4248-9f0b-df7c6d74616d admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:33.548058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db40114-9d42-4248-9f0b-df7c6d74616d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:33.553359 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 875/1747] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:33 2022] POST /identity/v3/users => generated 294 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:27:33.559237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ec3b7-5fd4-4c2c-8ee0-4787d1ddba32 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:33.566336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db40114-9d42-4248-9f0b-df7c6d74616d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:33.588049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83ec3b7-5fd4-4c2c-8ee0-4787d1ddba32 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:33.613890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b9ed5-9536-47d0-b723-02b4d2f8e185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:33.614721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b9ed5-9536-47d0-b723-02b4d2f8e185 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:33.615169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b9ed5-9536-47d0-b723-02b4d2f8e185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:33.617831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b9ed5-9536-47d0-b723-02b4d2f8e185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:33.629503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b9ed5-9536-47d0-b723-02b4d2f8e185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:33.637145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410b9ed5-9536-47d0-b723-02b4d2f8e185 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:33.648259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 873/1748] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:33 2022] POST /identity/v3/users => generated 292 bytes in 438 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:33.655157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db40114-9d42-4248-9f0b-df7c6d74616d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:33.686659 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 876/1749] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:33.694124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b9ed5-9536-47d0-b723-02b4d2f8e185 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:33.712140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db40114-9d42-4248-9f0b-df7c6d74616d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:33.729087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b9ed5-9536-47d0-b723-02b4d2f8e185 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:33.731364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc253e3-f552-46f3-930e-3b44be2cef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:33.732363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc253e3-f552-46f3-930e-3b44be2cef9c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:33.735005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc253e3-f552-46f3-930e-3b44be2cef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:33.735510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc253e3-f552-46f3-930e-3b44be2cef9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:33.744478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc253e3-f552-46f3-930e-3b44be2cef9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:33.746315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d931fa2-0771-4603-ba18-6b18c7c0b138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:33.747235 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d931fa2-0771-4603-ba18-6b18c7c0b138 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:33.747773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d931fa2-0771-4603-ba18-6b18c7c0b138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:33.747993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d931fa2-0771-4603-ba18-6b18c7c0b138 admin admin] PATH_INFO: `/v3/projects/4f3cdd00ffa446d1b7b79e2376b8e27e/users/4afe6a57c79a4c959ae81b0201540985/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:33.752957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc253e3-f552-46f3-930e-3b44be2cef9c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:33.788067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 874/1750] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:33.798238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc253e3-f552-46f3-930e-3b44be2cef9c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:33.814935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d931fa2-0771-4603-ba18-6b18c7c0b138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3cdd00ffa446d1b7b79e2376b8e27e, user_id=4afe6a57c79a4c959ae81b0201540985, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:33.826930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d931fa2-0771-4603-ba18-6b18c7c0b138 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:33.844169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc253e3-f552-46f3-930e-3b44be2cef9c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:33.854444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 877/1751] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:33 2022] PUT /identity/v3/projects/4f3cdd00ffa446d1b7b79e2376b8e27e/users/4afe6a57c79a4c959ae81b0201540985/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:33.858742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d931fa2-0771-4603-ba18-6b18c7c0b138 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:33.940956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:34.102561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:34.135282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cd68e-9eb3-4754-bea9-d3bb54d6932e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:34.136535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d931fa2-0771-4603-ba18-6b18c7c0b138 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:34.137459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cd68e-9eb3-4754-bea9-d3bb54d6932e admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:34.137886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cd68e-9eb3-4754-bea9-d3bb54d6932e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:34.139613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304cd68e-9eb3-4754-bea9-d3bb54d6932e admin admin] PATH_INFO: `/v3/projects/bcc9dd4c7a0949c98eae0de0ec1d8bfe/users/167ff5a8e9d64ac5863967c1de9fb071/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:34.145618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e29641-ba2d-45c3-82ff-7b9f42e3a0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:34.147386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e29641-ba2d-45c3-82ff-7b9f42e3a0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:34.147386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e29641-ba2d-45c3-82ff-7b9f42e3a0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:34.147802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e29641-ba2d-45c3-82ff-7b9f42e3a0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:34.150730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e29641-ba2d-45c3-82ff-7b9f42e3a0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:34.153594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e29641-ba2d-45c3-82ff-7b9f42e3a0a0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:34.174529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 878/1752] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:34.176056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e29641-ba2d-45c3-82ff-7b9f42e3a0a0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:34.179423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-74e29641-ba2d-45c3-82ff-7b9f42e3a0a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:34.183854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cd68e-9eb3-4754-bea9-d3bb54d6932e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc9dd4c7a0949c98eae0de0ec1d8bfe, user_id=167ff5a8e9d64ac5863967c1de9fb071, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:34.196059 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304cd68e-9eb3-4754-bea9-d3bb54d6932e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:34.216792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e29641-ba2d-45c3-82ff-7b9f42e3a0a0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:34.223202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 875/1753] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:33 2022] PUT /identity/v3/projects/bcc9dd4c7a0949c98eae0de0ec1d8bfe/users/167ff5a8e9d64ac5863967c1de9fb071/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:34.224171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cd68e-9eb3-4754-bea9-d3bb54d6932e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:34.234069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-304cd68e-9eb3-4754-bea9-d3bb54d6932e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:34.381505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:34.381750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:34.382044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:34.382327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] PATH_INFO: `/v3/projects/e93fc8565caa47a5b3128d76c3969127` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:34.383599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e93fc8565caa47a5b3128d76c3969127)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:34.386477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:34.445030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304cd68e-9eb3-4754-bea9-d3bb54d6932e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:34.448709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e93fc8565caa47a5b3128d76c3969127', 'request_id': 'req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:34.456458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e93fc8565caa47a5b3128d76c3969127', 'request_id': 'req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:34.471007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e93fc8565caa47a5b3128d76c3969127', 'request_id': 'req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:34.498936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] The token cache is being invalidate because project e93fc8565caa47a5b3128d76c3969127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:34.498936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:34.499954 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:34.505528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 879/1754] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:34 2022] DELETE /identity/v3/projects/e93fc8565caa47a5b3128d76c3969127 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:34.505528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:34.507293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:34.594049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e97119806774c2482a9886b11ab6f70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:34.594804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:34.597550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:34.598147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:34.602478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:34.622030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:34.631022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:34.633509 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:34.657849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 876/1755] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:34.660169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:34.662284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:34.663994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f875c1f-06cc-441b-8e58-e81d0ff7bed9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:34.682784 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b6f0c3-040c-4177-a98b-cbf74d13ed68 service placement] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:34.682876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:34.683676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:34.683973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:34.684259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] PATH_INFO: `/v3/projects/a2cc9d1b356e4e8aba32a63fbd11a985` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:34.691842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2cc9d1b356e4e8aba32a63fbd11a985)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:34.697246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:34.699692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c068644-5efa-4ee5-a4f2-15fbe24aa995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:34.701332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c068644-5efa-4ee5-a4f2-15fbe24aa995 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:34.701664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c068644-5efa-4ee5-a4f2-15fbe24aa995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:34.701946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c068644-5efa-4ee5-a4f2-15fbe24aa995 admin admin] PATH_INFO: `/v3/projects/4f3cdd00ffa446d1b7b79e2376b8e27e/users/4afe6a57c79a4c959ae81b0201540985/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:34.706705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c068644-5efa-4ee5-a4f2-15fbe24aa995 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f3cdd00ffa446d1b7b79e2376b8e27e, user_id=4afe6a57c79a4c959ae81b0201540985, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:34.714414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c068644-5efa-4ee5-a4f2-15fbe24aa995 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:34.757873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 877/1756] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:34 2022] PUT /identity/v3/projects/4f3cdd00ffa446d1b7b79e2376b8e27e/users/4afe6a57c79a4c959ae81b0201540985/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:34.757873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c068644-5efa-4ee5-a4f2-15fbe24aa995 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:34.758096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2c068644-5efa-4ee5-a4f2-15fbe24aa995 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:34.891205 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2cc9d1b356e4e8aba32a63fbd11a985', 'request_id': 'req-15e417fe-e480-45fe-ba62-569e674bfb9c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:34.946893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2cc9d1b356e4e8aba32a63fbd11a985', 'request_id': 'req-15e417fe-e480-45fe-ba62-569e674bfb9c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:34.955233 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:34.980359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c068644-5efa-4ee5-a4f2-15fbe24aa995 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:34.994623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2cc9d1b356e4e8aba32a63fbd11a985', 'request_id': 'req-15e417fe-e480-45fe-ba62-569e674bfb9c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:35.024355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] The token cache is being invalidate because project a2cc9d1b356e4e8aba32a63fbd11a985 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:35.025004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:35.026043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-15e417fe-e480-45fe-ba62-569e674bfb9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:35.028575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 880/1757] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:34 2022] DELETE /identity/v3/projects/a2cc9d1b356e4e8aba32a63fbd11a985 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:35.030874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b50c0-d426-4ea4-a363-1b3aaa3171e0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:35.031354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b50c0-d426-4ea4-a363-1b3aaa3171e0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:35.031843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6b50c0-d426-4ea4-a363-1b3aaa3171e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:35.154505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0a6b50c0-d426-4ea4-a363-1b3aaa3171e0 None None] MFA Rules not processed for user `8f448776d1074e22bfe6cfbe05e01db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:35.155775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0a6b50c0-d426-4ea4-a363-1b3aaa3171e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:35.286354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4de6a-5b23-45d9-91b9-a0500cbcce92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:35.287065 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4de6a-5b23-45d9-91b9-a0500cbcce92 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:35.287354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4de6a-5b23-45d9-91b9-a0500cbcce92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:35.287635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4de6a-5b23-45d9-91b9-a0500cbcce92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:35.290028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4de6a-5b23-45d9-91b9-a0500cbcce92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:35.292829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a4de6a-5b23-45d9-91b9-a0500cbcce92 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:35.344558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 878/1758] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:34 2022] GET /identity/v3/roles => generated 1420 bytes in 595 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:35.346767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4de6a-5b23-45d9-91b9-a0500cbcce92 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:35.351977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f3a4de6a-5b23-45d9-91b9-a0500cbcce92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:35.430911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a4de6a-5b23-45d9-91b9-a0500cbcce92 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:35.445276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:35.446713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:35.447149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:35.447929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] PATH_INFO: `/v3/users/6be665c86ead4d5ab71902e908d53747` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:35.449761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6be665c86ead4d5ab71902e908d53747)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:35.452165 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:35.467150 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 881/1759] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:35.472917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b2d2c-da8f-4e44-86ab-e57993f8ad09 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:35.473238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b2d2c-da8f-4e44-86ab-e57993f8ad09 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:35.476219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3b2d2c-da8f-4e44-86ab-e57993f8ad09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:35.593067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8a3b2d2c-da8f-4e44-86ab-e57993f8ad09 None None] MFA Rules not processed for user `4afe6a57c79a4c959ae81b0201540985`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:35.595147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8a3b2d2c-da8f-4e44-86ab-e57993f8ad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:35.616517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6be665c86ead4d5ab71902e908d53747'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:35.643969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6be665c86ead4d5ab71902e908d53747'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:35.649889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6be665c86ead4d5ab71902e908d53747'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:35.676510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6be665c86ead4d5ab71902e908d53747'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:35.745203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6be665c86ead4d5ab71902e908d53747'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:35.753167 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 879/1760] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:35 2022] DELETE /identity/v3/users/6be665c86ead4d5ab71902e908d53747 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:35.754202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:35.760319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:35.811714 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:35.859617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 882/1761] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:35 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:35.859617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b2d2c-da8f-4e44-86ab-e57993f8ad09 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:35.859617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8a3b2d2c-da8f-4e44-86ab-e57993f8ad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:35.915352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3b2d2c-da8f-4e44-86ab-e57993f8ad09 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:35.970833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0920fe-baa3-4256-8afd-3b1bed2a1a69 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:35.985875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:35.987445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:35.987775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:35.988102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] PATH_INFO: `/v3/users/3603bc39008f4c8199e13e54d8832205` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:35.990210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3603bc39008f4c8199e13e54d8832205)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:35.992210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:36.022349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:36.128915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b818a613-e7d3-47e1-931f-bf3b54f3e506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:36.131943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818a613-e7d3-47e1-931f-bf3b54f3e506 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:36.132240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818a613-e7d3-47e1-931f-bf3b54f3e506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:36.132530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b818a613-e7d3-47e1-931f-bf3b54f3e506 admin admin] PATH_INFO: `/v3/projects/bcc9dd4c7a0949c98eae0de0ec1d8bfe/users/167ff5a8e9d64ac5863967c1de9fb071/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:36.137068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818a613-e7d3-47e1-931f-bf3b54f3e506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc9dd4c7a0949c98eae0de0ec1d8bfe, user_id=167ff5a8e9d64ac5863967c1de9fb071, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:36.140874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b818a613-e7d3-47e1-931f-bf3b54f3e506 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:36.159130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3603bc39008f4c8199e13e54d8832205'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:36.170285 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3603bc39008f4c8199e13e54d8832205'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:36.171354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3603bc39008f4c8199e13e54d8832205'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:36.180239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 883/1762] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:35 2022] PUT /identity/v3/projects/bcc9dd4c7a0949c98eae0de0ec1d8bfe/users/167ff5a8e9d64ac5863967c1de9fb071/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:36.184891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3603bc39008f4c8199e13e54d8832205'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:36.192017 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6578910-aacf-4158-840f-9dd5a01ac078 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:36.192502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6578910-aacf-4158-840f-9dd5a01ac078 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:36.192952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6578910-aacf-4158-840f-9dd5a01ac078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:36.208264 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d94c798-471d-4ea6-a3db-e2dd5960d677 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3603bc39008f4c8199e13e54d8832205'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:36.241268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f6578910-aacf-4158-840f-9dd5a01ac078 None None] MFA Rules not processed for user `8f448776d1074e22bfe6cfbe05e01db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:36.242123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 880/1763] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:35 2022] DELETE /identity/v3/users/3603bc39008f4c8199e13e54d8832205 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:36.246434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f6578910-aacf-4158-840f-9dd5a01ac078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:36.253875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45952a-b5a5-4917-bc3c-01ee81432835 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:36.254173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45952a-b5a5-4917-bc3c-01ee81432835 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:36.254506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e45952a-b5a5-4917-bc3c-01ee81432835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:36.324029 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1e45952a-b5a5-4917-bc3c-01ee81432835 None None] MFA Rules not processed for user `83bbc8ad40b44661b526511ecbc51027`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:36.326184 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1e45952a-b5a5-4917-bc3c-01ee81432835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:36.412428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 884/1764] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:36.413475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6578910-aacf-4158-840f-9dd5a01ac078 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:36.416247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f6578910-aacf-4158-840f-9dd5a01ac078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:36.496088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 881/1765] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:36 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:36.497721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed0da5-fbc7-4f1c-b6dc-8b817e6e003e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:36.498010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed0da5-fbc7-4f1c-b6dc-8b817e6e003e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:36.499610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ed0da5-fbc7-4f1c-b6dc-8b817e6e003e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:36.546116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6578910-aacf-4158-840f-9dd5a01ac078 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:36.550711 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:36.551613 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:36.551982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:36.552388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:36.552496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e9ed0da5-fbc7-4f1c-b6dc-8b817e6e003e None None] MFA Rules not processed for user `167ff5a8e9d64ac5863967c1de9fb071`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:36.554682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e9ed0da5-fbc7-4f1c-b6dc-8b817e6e003e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:36.555706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:36.619251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:36.649745 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:36.653967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:36.674155 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 885/1766] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:36.676753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:36.683670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:36.766108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:36.779356 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 882/1767] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:36.784325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d36f3-e790-43e2-a005-ca69e748db1c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:36.785140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d36f3-e790-43e2-a005-ca69e748db1c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:36.786037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8d36f3-e790-43e2-a005-ca69e748db1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:36.869051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-fd8d36f3-e790-43e2-a005-ca69e748db1c None None] MFA Rules not processed for user `c4b710bbdf0c4d58b4f3ee7ef7410c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:36.872434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fd8d36f3-e790-43e2-a005-ca69e748db1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:37.010083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999ae20d-3109-4e02-988c-3930bdbe7f1b service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:37.033099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:37.034191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:37.035833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:37.036253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:37.040162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:37.059628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 883/1768] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:37.063200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf39f9-26ef-4b88-8733-f14ad619a1d8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:37.063697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf39f9-26ef-4b88-8733-f14ad619a1d8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:37.065736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddf39f9-26ef-4b88-8733-f14ad619a1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:37.076940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:37.079536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:37.110014 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 886/1769] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:36 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:37.112211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:37.115284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-2ddf39f9-26ef-4b88-8733-f14ad619a1d8 None None] MFA Rules not processed for user `83bbc8ad40b44661b526511ecbc51027`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:37.117779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:37.124137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2ddf39f9-26ef-4b88-8733-f14ad619a1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:37.145877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce54026-8938-49bf-9dbd-a7dcaa5fed87 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:37.159966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 884/1770] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:37.166504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf39f9-26ef-4b88-8733-f14ad619a1d8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:37.169480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7931be-dba8-4bd3-adf8-664ff292df8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:37.170123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7931be-dba8-4bd3-adf8-664ff292df8f service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:37.170534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7931be-dba8-4bd3-adf8-664ff292df8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:37.170924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7931be-dba8-4bd3-adf8-664ff292df8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:37.171824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2ddf39f9-26ef-4b88-8733-f14ad619a1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:37.192207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7931be-dba8-4bd3-adf8-664ff292df8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:37.196136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7931be-dba8-4bd3-adf8-664ff292df8f service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:37.221209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 887/1771] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:37 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:37.338489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddf39f9-26ef-4b88-8733-f14ad619a1d8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:37.353134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:37.354218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:37.354518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:37.354781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] PATH_INFO: `/v3/projects/a6fb4f8909554b928018143e2017c73f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:37.358506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6fb4f8909554b928018143e2017c73f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:37.361782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:37.505369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6fb4f8909554b928018143e2017c73f', 'request_id': 'req-30967d4e-714c-4b85-a583-7c7859a826b3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:37.522770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6fb4f8909554b928018143e2017c73f', 'request_id': 'req-30967d4e-714c-4b85-a583-7c7859a826b3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:37.547517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6fb4f8909554b928018143e2017c73f', 'request_id': 'req-30967d4e-714c-4b85-a583-7c7859a826b3'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:37.582084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] The token cache is being invalidate because project a6fb4f8909554b928018143e2017c73f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:37.582388 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:37.585016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-30967d4e-714c-4b85-a583-7c7859a826b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:37.589780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7931be-dba8-4bd3-adf8-664ff292df8f service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:37.591456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9c7931be-dba8-4bd3-adf8-664ff292df8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:37.593540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 885/1772] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:37 2022] DELETE /identity/v3/projects/a6fb4f8909554b928018143e2017c73f => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:37.595041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a681eb-a90c-440b-8892-c3c901dbcd5d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:37.595389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a681eb-a90c-440b-8892-c3c901dbcd5d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:37.595677 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a681eb-a90c-440b-8892-c3c901dbcd5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:37.628164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7931be-dba8-4bd3-adf8-664ff292df8f service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:37.635524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-57a681eb-a90c-440b-8892-c3c901dbcd5d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:37.636957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-57a681eb-a90c-440b-8892-c3c901dbcd5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:37.744016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:37.803689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:37.804704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:37.808596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:37.810089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] PATH_INFO: `/v3/projects/7f30a18ba5c646868927ec9cdd286614` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:37.812629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f30a18ba5c646868927ec9cdd286614)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:37.818670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:37.861955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 886/1773] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:37.871191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc9446-fff9-4c5f-8c6f-bc60b3292ff1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:37.871849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc9446-fff9-4c5f-8c6f-bc60b3292ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:37.872282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bc9446-fff9-4c5f-8c6f-bc60b3292ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:37.947111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e2bc9446-fff9-4c5f-8c6f-bc60b3292ff1 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:37.949490 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e2bc9446-fff9-4c5f-8c6f-bc60b3292ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:37.967363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f30a18ba5c646868927ec9cdd286614', 'request_id': 'req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:37.985957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f30a18ba5c646868927ec9cdd286614', 'request_id': 'req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:37.990841 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:38.007368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f30a18ba5c646868927ec9cdd286614', 'request_id': 'req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:38.042833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] The token cache is being invalidate because project 7f30a18ba5c646868927ec9cdd286614 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:38.043139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:38.045840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eafb1a85-4631-4f3a-a91d-8d3c4df8bce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:38.055655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 888/1774] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:37 2022] DELETE /identity/v3/projects/7f30a18ba5c646868927ec9cdd286614 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:38.058720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6045ca-6ae8-442d-bf6c-4f4f56d07d10 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.058720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6045ca-6ae8-442d-bf6c-4f4f56d07d10 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.058972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6045ca-6ae8-442d-bf6c-4f4f56d07d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.083237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 887/1775] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:27:38.088199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc9446-fff9-4c5f-8c6f-bc60b3292ff1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:38.114617 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0a6045ca-6ae8-442d-bf6c-4f4f56d07d10 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:38.115952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0a6045ca-6ae8-442d-bf6c-4f4f56d07d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:38.126327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bc9446-fff9-4c5f-8c6f-bc60b3292ff1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:38.139814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 889/1776] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:38.144713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:38.146059 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.147220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.147523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.148113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d8301-f520-4d9b-a4f0-81e65c27dcc5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.148413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d8301-f520-4d9b-a4f0-81e65c27dcc5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.148695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520d8301-f520-4d9b-a4f0-81e65c27dcc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.150061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:38.150403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:38.159336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:38.202626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-520d8301-f520-4d9b-a4f0-81e65c27dcc5 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:38.204102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-520d8301-f520-4d9b-a4f0-81e65c27dcc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:38.210411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 888/1777] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:38.211354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:38.232496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:38.234367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 890/1778] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:27:38.236488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d8301-f520-4d9b-a4f0-81e65c27dcc5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:38.268171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520d8301-f520-4d9b-a4f0-81e65c27dcc5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:38.278135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46952160-ec15-4909-9545-f8c303a86810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:38.281257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46952160-ec15-4909-9545-f8c303a86810 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.285103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46952160-ec15-4909-9545-f8c303a86810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.286779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46952160-ec15-4909-9545-f8c303a86810 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.287838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46952160-ec15-4909-9545-f8c303a86810 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:38.290704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46952160-ec15-4909-9545-f8c303a86810 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:38.296396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692968e7-0231-47ad-b4e4-40b147d57a37 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:38.317171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:38.318155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.318448 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.318737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] PATH_INFO: `/v3/users/4863b2cc79584ba5932770056a47f81c` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.323859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4863b2cc79584ba5932770056a47f81c)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:38.333554 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:38.377549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 891/1779] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:38 2022] POST /identity/v3/projects => generated 379 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:38.377549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46952160-ec15-4909-9545-f8c303a86810 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:38.481753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4863b2cc79584ba5932770056a47f81c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:38.510704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4863b2cc79584ba5932770056a47f81c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:38.511783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4863b2cc79584ba5932770056a47f81c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:38.551024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4863b2cc79584ba5932770056a47f81c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:38.586215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4863b2cc79584ba5932770056a47f81c'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:38.607768 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 889/1780] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:38 2022] DELETE /identity/v3/users/4863b2cc79584ba5932770056a47f81c => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:38.611719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:38.617548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:38.618525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46952160-ec15-4909-9545-f8c303a86810 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:38.709063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee29c75-f2ee-4cba-9342-e13e71c42e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:38.710063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee29c75-f2ee-4cba-9342-e13e71c42e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.710491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee29c75-f2ee-4cba-9342-e13e71c42e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.710908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee29c75-f2ee-4cba-9342-e13e71c42e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.712457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee29c75-f2ee-4cba-9342-e13e71c42e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:38.713356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee29c75-f2ee-4cba-9342-e13e71c42e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:38.715186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee29c75-f2ee-4cba-9342-e13e71c42e4b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:38.718063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:38.731242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 892/1781] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:38.733470 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e9d77-97e0-4dda-a28b-1da6fa7a9f24 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.734017 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e9d77-97e0-4dda-a28b-1da6fa7a9f24 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.734640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1e9d77-97e0-4dda-a28b-1da6fa7a9f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.770389 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cd1e9d77-97e0-4dda-a28b-1da6fa7a9f24 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:38.773112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cd1e9d77-97e0-4dda-a28b-1da6fa7a9f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:38.822057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91313b20-343a-49a6-a410-53fc87f2a62e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:38.839862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 893/1782] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:38.843069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89a2c1-c9d1-4150-b2a9-acee06411b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:38.844105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89a2c1-c9d1-4150-b2a9-acee06411b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.844523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89a2c1-c9d1-4150-b2a9-acee06411b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.844959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89a2c1-c9d1-4150-b2a9-acee06411b55 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.846125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e9d77-97e0-4dda-a28b-1da6fa7a9f24 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:38.846263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89a2c1-c9d1-4150-b2a9-acee06411b55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:38.852765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cd1e9d77-97e0-4dda-a28b-1da6fa7a9f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:38.854682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89a2c1-c9d1-4150-b2a9-acee06411b55 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:38.884422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1e9d77-97e0-4dda-a28b-1da6fa7a9f24 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:38.894769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17952443-64ec-48f2-a0ad-7b0d73ba3b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:38.898332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17952443-64ec-48f2-a0ad-7b0d73ba3b4b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:38.898812 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17952443-64ec-48f2-a0ad-7b0d73ba3b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:38.899231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17952443-64ec-48f2-a0ad-7b0d73ba3b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:38.900131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17952443-64ec-48f2-a0ad-7b0d73ba3b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:38.916497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17952443-64ec-48f2-a0ad-7b0d73ba3b4b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:38.944143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 890/1783] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:38 2022] POST /identity/v3/users => generated 290 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:38.945362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89a2c1-c9d1-4150-b2a9-acee06411b55 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:38.952707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4e89a2c1-c9d1-4150-b2a9-acee06411b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:39.009885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 894/1784] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:38 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:39.011782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17952443-64ec-48f2-a0ad-7b0d73ba3b4b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:39.098509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:39.179825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89a2c1-c9d1-4150-b2a9-acee06411b55 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:39.240704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17952443-64ec-48f2-a0ad-7b0d73ba3b4b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:39.254137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68295426-4262-42c9-99fe-4afe9da1bab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:39.254829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68295426-4262-42c9-99fe-4afe9da1bab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.255118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68295426-4262-42c9-99fe-4afe9da1bab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.255396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68295426-4262-42c9-99fe-4afe9da1bab7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.256135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68295426-4262-42c9-99fe-4afe9da1bab7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:39.257825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68295426-4262-42c9-99fe-4afe9da1bab7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:39.262993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc04cb-c044-4ea8-8144-d885a0a624ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:39.263679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc04cb-c044-4ea8-8144-d885a0a624ba service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.263966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc04cb-c044-4ea8-8144-d885a0a624ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.264276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc04cb-c044-4ea8-8144-d885a0a624ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.279365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc04cb-c044-4ea8-8144-d885a0a624ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:39.281079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc04cb-c044-4ea8-8144-d885a0a624ba service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:39.294005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 891/1785] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:39.296434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627081b5-c36c-4f74-aae6-aa999723bc4f None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.296919 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627081b5-c36c-4f74-aae6-aa999723bc4f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.297471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627081b5-c36c-4f74-aae6-aa999723bc4f None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.300074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 892/1786] 10.222.0.23 () {54 vars in 912 bytes} [Tue Jul 12 13:27:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:39.301727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0a974-068c-4957-a6c7-d11c5e404927 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.302025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0a974-068c-4957-a6c7-d11c5e404927 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.302305 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a0a974-068c-4957-a6c7-d11c5e404927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.336737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 895/1787] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:39.337733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68295426-4262-42c9-99fe-4afe9da1bab7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:39.372399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68295426-4262-42c9-99fe-4afe9da1bab7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:39.394630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-12a0a974-068c-4957-a6c7-d11c5e404927 None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:39.396363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87cdd38-9a47-4299-87e3-01f19aa5b814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:39.396605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-12a0a974-068c-4957-a6c7-d11c5e404927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:39.398934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cdd38-9a47-4299-87e3-01f19aa5b814 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.399366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cdd38-9a47-4299-87e3-01f19aa5b814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.399760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87cdd38-9a47-4299-87e3-01f19aa5b814 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.402339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87cdd38-9a47-4299-87e3-01f19aa5b814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:39.408976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87cdd38-9a47-4299-87e3-01f19aa5b814 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:39.468356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 896/1788] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:39 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:39.469939 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1837-a148-4d62-8f1b-bdb9a0e51ec3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.470094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1837-a148-4d62-8f1b-bdb9a0e51ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.473457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b1837-a148-4d62-8f1b-bdb9a0e51ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.518566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ea4b1837-a148-4d62-8f1b-bdb9a0e51ec3 None None] MFA Rules not processed for user `83837e4f8f9b4e538658876cfb56db16`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:39.520024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ea4b1837-a148-4d62-8f1b-bdb9a0e51ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:39.521974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:39.559233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 897/1789] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:27:39 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:39.560113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b1837-a148-4d62-8f1b-bdb9a0e51ec3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:39.566426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ea4b1837-a148-4d62-8f1b-bdb9a0e51ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:39.577020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 893/1790] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:27:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:39.598842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0a974-068c-4957-a6c7-d11c5e404927 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:39.640266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a0a974-068c-4957-a6c7-d11c5e404927 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:39.653585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7518823-b8be-4414-8c62-7730df0a679f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:39.654540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7518823-b8be-4414-8c62-7730df0a679f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.654978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7518823-b8be-4414-8c62-7730df0a679f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.655436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7518823-b8be-4414-8c62-7730df0a679f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.656839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7518823-b8be-4414-8c62-7730df0a679f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:39.662434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7518823-b8be-4414-8c62-7730df0a679f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:39.679990 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 894/1791] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:39.681178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7518823-b8be-4414-8c62-7730df0a679f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:39.699590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7518823-b8be-4414-8c62-7730df0a679f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:39.715965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20498392-9487-4a8c-a0e8-eca74d8d8db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:39.716672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20498392-9487-4a8c-a0e8-eca74d8d8db4 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.716997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20498392-9487-4a8c-a0e8-eca74d8d8db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.717401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20498392-9487-4a8c-a0e8-eca74d8d8db4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.718180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20498392-9487-4a8c-a0e8-eca74d8d8db4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:39.727033 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20498392-9487-4a8c-a0e8-eca74d8d8db4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:39.772965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 895/1792] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:39.775026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20498392-9487-4a8c-a0e8-eca74d8d8db4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:39.783430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-20498392-9487-4a8c-a0e8-eca74d8d8db4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:39.798014 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b1837-a148-4d62-8f1b-bdb9a0e51ec3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:39.815210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20498392-9487-4a8c-a0e8-eca74d8d8db4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:39.815722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:39.817054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.817458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.817799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.822999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:39.827647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685ac90-05c1-487e-b34b-79767240b92c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:39.828405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685ac90-05c1-487e-b34b-79767240b92c service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.828626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685ac90-05c1-487e-b34b-79767240b92c service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.831063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8685ac90-05c1-487e-b34b-79767240b92c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.855095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685ac90-05c1-487e-b34b-79767240b92c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:39.856853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8685ac90-05c1-487e-b34b-79767240b92c service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:39.878048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 896/1793] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:39.878842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685ac90-05c1-487e-b34b-79767240b92c service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:39.908672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8685ac90-05c1-487e-b34b-79767240b92c service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:39.925911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f19897-1f03-4ecb-a5c6-e0eb823bfb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:39.926855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f19897-1f03-4ecb-a5c6-e0eb823bfb26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:39.927185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f19897-1f03-4ecb-a5c6-e0eb823bfb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:39.927471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f19897-1f03-4ecb-a5c6-e0eb823bfb26 admin admin] PATH_INFO: `/v3/projects/4f652ccd643c4a9db402892e69ff67e2/users/ba3f064fa2894265b1f9072e717b550d/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:39.969335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f19897-1f03-4ecb-a5c6-e0eb823bfb26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f652ccd643c4a9db402892e69ff67e2, user_id=ba3f064fa2894265b1f9072e717b550d, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:39.973172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f19897-1f03-4ecb-a5c6-e0eb823bfb26 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:40.002643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 897/1794] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:39 2022] PUT /identity/v3/projects/4f652ccd643c4a9db402892e69ff67e2/users/ba3f064fa2894265b1f9072e717b550d/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:40.003770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f19897-1f03-4ecb-a5c6-e0eb823bfb26 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:40.139373 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:40.146610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:40.163759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 898/1795] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:40.165441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:40.167007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:40.219346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f19897-1f03-4ecb-a5c6-e0eb823bfb26 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:40.239546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ed6a9-82db-4f24-9ef4-4b87b0d31f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:40.240703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed6a9-82db-4f24-9ef4-4b87b0d31f0a admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:40.241054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed6a9-82db-4f24-9ef4-4b87b0d31f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:40.241339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ed6a9-82db-4f24-9ef4-4b87b0d31f0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:40.242142 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ed6a9-82db-4f24-9ef4-4b87b0d31f0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:40.254904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ed6a9-82db-4f24-9ef4-4b87b0d31f0a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:40.313956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 898/1796] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:40.315786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d20816-1677-4f37-b1d1-6966ba8fba7a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:40.316371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d20816-1677-4f37-b1d1-6966ba8fba7a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:40.316777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d20816-1677-4f37-b1d1-6966ba8fba7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:40.319551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:40.375971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-34d20816-1677-4f37-b1d1-6966ba8fba7a None None] MFA Rules not processed for user `ba3f064fa2894265b1f9072e717b550d`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:40.378178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-34d20816-1677-4f37-b1d1-6966ba8fba7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:40.448653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414929e2-10ac-4b1e-b17d-613e67ba1f38 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:40.604333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 899/1797] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:40.609222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d20816-1677-4f37-b1d1-6966ba8fba7a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:40.639136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d20816-1677-4f37-b1d1-6966ba8fba7a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:40.641805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87070dcb-0418-49cc-85df-c38d715fa77a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:40.643280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87070dcb-0418-49cc-85df-c38d715fa77a service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:40.644089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87070dcb-0418-49cc-85df-c38d715fa77a service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:40.644708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87070dcb-0418-49cc-85df-c38d715fa77a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:40.646206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525f9c9-faca-4fbf-a370-a85a5bacd88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:40.647417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525f9c9-faca-4fbf-a370-a85a5bacd88d admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:40.648009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525f9c9-faca-4fbf-a370-a85a5bacd88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:40.648300 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7525f9c9-faca-4fbf-a370-a85a5bacd88d admin admin] PATH_INFO: `/v3/projects/8680e2820f5441daa1fc34b008a96684/users/0372d04ff5604d07a5af4e0ec9c23b1d/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:40.682739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:40.695447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525f9c9-faca-4fbf-a370-a85a5bacd88d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8680e2820f5441daa1fc34b008a96684, user_id=0372d04ff5604d07a5af4e0ec9c23b1d, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:40.705949 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7525f9c9-faca-4fbf-a370-a85a5bacd88d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:40.749033 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 900/1798] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:40 2022] PUT /identity/v3/projects/8680e2820f5441daa1fc34b008a96684/users/0372d04ff5604d07a5af4e0ec9c23b1d/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:40.749914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525f9c9-faca-4fbf-a370-a85a5bacd88d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:40.836827 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87070dcb-0418-49cc-85df-c38d715fa77a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:40.839180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87070dcb-0418-49cc-85df-c38d715fa77a service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:40.875705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 899/1799] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:27:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 711 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:27:40.886830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8c378-de94-4714-a176-5adc7763bc44 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:40.887572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8c378-de94-4714-a176-5adc7763bc44 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:40.889976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b8c378-de94-4714-a176-5adc7763bc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:41.013276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a0b8c378-de94-4714-a176-5adc7763bc44 None None] MFA Rules not processed for user `0372d04ff5604d07a5af4e0ec9c23b1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:41.014722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a0b8c378-de94-4714-a176-5adc7763bc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:41.034595 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7525f9c9-faca-4fbf-a370-a85a5bacd88d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:41.062454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4026570-5cad-4a84-b5c6-244e975ab4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:41.066011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4026570-5cad-4a84-b5c6-244e975ab4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:41.072940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4026570-5cad-4a84-b5c6-244e975ab4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:41.073861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4026570-5cad-4a84-b5c6-244e975ab4d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:41.074700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4026570-5cad-4a84-b5c6-244e975ab4d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:41.082074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4026570-5cad-4a84-b5c6-244e975ab4d1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:41.209435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 901/1800] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:40 2022] POST /identity/v3/projects => generated 379 bytes in 457 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:41.209435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4026570-5cad-4a84-b5c6-244e975ab4d1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:41.211542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b4026570-5cad-4a84-b5c6-244e975ab4d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:41.259978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:41.316004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4026570-5cad-4a84-b5c6-244e975ab4d1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:41.321078 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 900/1801] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:41.321934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8c378-de94-4714-a176-5adc7763bc44 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:41.339485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:41.340924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:41.341497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:41.342236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b8c378-de94-4714-a176-5adc7763bc44 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:41.342465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] PATH_INFO: `/v3/projects/dd4b3ed2049c42ea97e262b5280f0f5a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:41.345037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd4b3ed2049c42ea97e262b5280f0f5a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:41.349097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:41.361879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69facaab-8ac3-4640-9484-8f5c1628be11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:41.362904 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69facaab-8ac3-4640-9484-8f5c1628be11 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:41.363287 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69facaab-8ac3-4640-9484-8f5c1628be11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:41.363573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69facaab-8ac3-4640-9484-8f5c1628be11 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:41.364666 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69facaab-8ac3-4640-9484-8f5c1628be11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:41.367598 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69facaab-8ac3-4640-9484-8f5c1628be11 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:41.422381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 901/1802] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:41 2022] POST /identity/v3/users => generated 296 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:41.424866 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69facaab-8ac3-4640-9484-8f5c1628be11 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:41.431055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-69facaab-8ac3-4640-9484-8f5c1628be11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:41.606818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd4b3ed2049c42ea97e262b5280f0f5a', 'request_id': 'req-5882ba01-a58d-44c5-b393-c779214bcdc5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:41.636306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd4b3ed2049c42ea97e262b5280f0f5a', 'request_id': 'req-5882ba01-a58d-44c5-b393-c779214bcdc5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:41.638539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69facaab-8ac3-4640-9484-8f5c1628be11 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:41.678855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd4b3ed2049c42ea97e262b5280f0f5a', 'request_id': 'req-5882ba01-a58d-44c5-b393-c779214bcdc5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:41.700821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] The token cache is being invalidate because project dd4b3ed2049c42ea97e262b5280f0f5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:41.700979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:41.704590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:41.718966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 902/1803] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:41 2022] DELETE /identity/v3/projects/dd4b3ed2049c42ea97e262b5280f0f5a => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:41.719987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:41.727226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:41.761287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:41.852441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:41.853298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:41.853749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:41.854174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:41.860110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:42.064742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5882ba01-a58d-44c5-b393-c779214bcdc5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:42.090091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:42.090692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31368b-9b46-43a8-8c13-2f11a7d39eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:42.091409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31368b-9b46-43a8-8c13-2f11a7d39eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:42.091767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31368b-9b46-43a8-8c13-2f11a7d39eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:42.092044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31368b-9b46-43a8-8c13-2f11a7d39eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:42.092824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31368b-9b46-43a8-8c13-2f11a7d39eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:42.093467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:42.094522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31368b-9b46-43a8-8c13-2f11a7d39eb7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:42.137753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 902/1804] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 715 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:42.138935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:42.147311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:42.181821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 903/1805] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 462 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:42.182795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31368b-9b46-43a8-8c13-2f11a7d39eb7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:42.232653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31368b-9b46-43a8-8c13-2f11a7d39eb7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:42.243037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa089f9-f724-424f-a338-207b367b7ab6 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:42.270739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6badf90-1794-4658-828b-53da592db65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:42.271677 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6badf90-1794-4658-828b-53da592db65e admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:42.277070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6badf90-1794-4658-828b-53da592db65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:42.277522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6badf90-1794-4658-828b-53da592db65e admin admin] PATH_INFO: `/v3/projects/4432bdb4b8774e7da14053179c90ca58/users/76b433c680bd4bba95065e344c15855b/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:42.287605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:42.296303 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:42.296676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:42.296970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] PATH_INFO: `/v3/users/167ff5a8e9d64ac5863967c1de9fb071` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:42.309679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] RBAC: Authorizing `identity:delete_user(user_id=167ff5a8e9d64ac5863967c1de9fb071)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:42.352314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6badf90-1794-4658-828b-53da592db65e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4432bdb4b8774e7da14053179c90ca58, user_id=76b433c680bd4bba95065e344c15855b, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:42.360687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:42.367040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6badf90-1794-4658-828b-53da592db65e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:42.418872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 904/1806] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:42 2022] PUT /identity/v3/projects/4432bdb4b8774e7da14053179c90ca58/users/76b433c680bd4bba95065e344c15855b/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:42.424752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6badf90-1794-4658-828b-53da592db65e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:42.566967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6badf90-1794-4658-828b-53da592db65e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:42.575854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f170b79-7948-4fe0-a03a-b986cba72a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:42.575854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f170b79-7948-4fe0-a03a-b986cba72a78 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:42.576381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f170b79-7948-4fe0-a03a-b986cba72a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:42.576543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f170b79-7948-4fe0-a03a-b986cba72a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:42.577562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f170b79-7948-4fe0-a03a-b986cba72a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:42.579687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f170b79-7948-4fe0-a03a-b986cba72a78 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:42.580891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:42.594897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 905/1807] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:42.597242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f170b79-7948-4fe0-a03a-b986cba72a78 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:42.610791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f170b79-7948-4fe0-a03a-b986cba72a78 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:42.618793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7203b83e-7eab-42b7-9281-b05db69a4488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:42.620266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203b83e-7eab-42b7-9281-b05db69a4488 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:42.620694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203b83e-7eab-42b7-9281-b05db69a4488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:42.621118 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7203b83e-7eab-42b7-9281-b05db69a4488 admin admin] PATH_INFO: `/v3/projects/4432bdb4b8774e7da14053179c90ca58/users/76b433c680bd4bba95065e344c15855b/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:42.625445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203b83e-7eab-42b7-9281-b05db69a4488 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4432bdb4b8774e7da14053179c90ca58, user_id=76b433c680bd4bba95065e344c15855b, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:42.633037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7203b83e-7eab-42b7-9281-b05db69a4488 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:42.645702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167ff5a8e9d64ac5863967c1de9fb071'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:42.650809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 906/1808] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:42 2022] PUT /identity/v3/projects/4432bdb4b8774e7da14053179c90ca58/users/76b433c680bd4bba95065e344c15855b/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:42.663130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015540c-7330-4c89-b52b-e0ae76b4a999 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:42.663538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015540c-7330-4c89-b52b-e0ae76b4a999 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:42.663826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e015540c-7330-4c89-b52b-e0ae76b4a999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:42.673268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167ff5a8e9d64ac5863967c1de9fb071'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:42.674928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167ff5a8e9d64ac5863967c1de9fb071'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:42.719016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167ff5a8e9d64ac5863967c1de9fb071'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:42.741754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e015540c-7330-4c89-b52b-e0ae76b4a999 None None] MFA Rules not processed for user `76b433c680bd4bba95065e344c15855b`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:42.741953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ecb034a4-48ed-4921-9b1b-4ed845f85502 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167ff5a8e9d64ac5863967c1de9fb071'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:42.744459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e015540c-7330-4c89-b52b-e0ae76b4a999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:42.750930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 903/1809] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:42 2022] DELETE /identity/v3/users/167ff5a8e9d64ac5863967c1de9fb071 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:42.780809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f36ba-7832-45f3-b19a-f08ef374b550 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:42.781255 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f36ba-7832-45f3-b19a-f08ef374b550 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:42.781706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f36ba-7832-45f3-b19a-f08ef374b550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:42.848449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:42.867918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-754f36ba-7832-45f3-b19a-f08ef374b550 None None] MFA Rules not processed for user `8f448776d1074e22bfe6cfbe05e01db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:42.872841 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-754f36ba-7832-45f3-b19a-f08ef374b550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:42.885428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 907/1810] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:42 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:42.900303 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015540c-7330-4c89-b52b-e0ae76b4a999 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:42.902293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e015540c-7330-4c89-b52b-e0ae76b4a999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:43.075545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 904/1811] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:43.087994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e015540c-7330-4c89-b52b-e0ae76b4a999 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:43.239344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7cc94-cc92-45dd-a204-29173bd9c11a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:43.240020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7cc94-cc92-45dd-a204-29173bd9c11a service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:43.240304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7cc94-cc92-45dd-a204-29173bd9c11a service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:43.240579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf7cc94-cc92-45dd-a204-29173bd9c11a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:43.338458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2a071-8cbf-4e3c-828f-81acd239c42b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:43.338774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2a071-8cbf-4e3c-828f-81acd239c42b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:43.339093 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa2a071-8cbf-4e3c-828f-81acd239c42b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:43.435463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bfa2a071-8cbf-4e3c-828f-81acd239c42b None None] MFA Rules not processed for user `83837e4f8f9b4e538658876cfb56db16`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:43.444113 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bfa2a071-8cbf-4e3c-828f-81acd239c42b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:43.494454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7cc94-cc92-45dd-a204-29173bd9c11a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:43.495994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf7cc94-cc92-45dd-a204-29173bd9c11a service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:43.522958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 908/1812] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:42 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 624 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:27:43.733348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 905/1813] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Jul 12 13:27:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:43.752658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7cc94-cc92-45dd-a204-29173bd9c11a service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:43.756460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9bf7cc94-cc92-45dd-a204-29173bd9c11a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:43.778240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf7cc94-cc92-45dd-a204-29173bd9c11a service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:43.797991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32062897-4185-405e-92d3-635af3d429f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:43.799662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32062897-4185-405e-92d3-635af3d429f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:43.801063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32062897-4185-405e-92d3-635af3d429f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:43.802116 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32062897-4185-405e-92d3-635af3d429f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:43.821646 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32062897-4185-405e-92d3-635af3d429f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:43.823763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32062897-4185-405e-92d3-635af3d429f0 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:43.869889 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2a071-8cbf-4e3c-828f-81acd239c42b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:43.885723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa2a071-8cbf-4e3c-828f-81acd239c42b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:43.901791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 909/1814] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:43.915990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:43.918067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:43.918563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:43.919809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:43.924251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:44.202160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:44.206540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:44.250286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:44.264445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 906/1815] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:27:44.603014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32062897-4185-405e-92d3-635af3d429f0 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:44.627714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32062897-4185-405e-92d3-635af3d429f0 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:44.641393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bef51-bc43-436d-8a0e-a3139ce89908 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b433c680bd4bba95065e344c15855b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4432bdb4b8774e7da14053179c90ca58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:44.643350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bef51-bc43-436d-8a0e-a3139ce89908 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:44.645516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bef51-bc43-436d-8a0e-a3139ce89908 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:44.646644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bef51-bc43-436d-8a0e-a3139ce89908 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:44.647587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bef51-bc43-436d-8a0e-a3139ce89908 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:44.649811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bef51-bc43-436d-8a0e-a3139ce89908 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:44.652241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bef51-bc43-436d-8a0e-a3139ce89908 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:44.710830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:44.711849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 910/1816] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:44.713659 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:44.718801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bef51-bc43-436d-8a0e-a3139ce89908 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:44.797662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bef51-bc43-436d-8a0e-a3139ce89908 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:44.851737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa4199-a8ec-45fb-9e11-09f5eb081927 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b433c680bd4bba95065e344c15855b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4432bdb4b8774e7da14053179c90ca58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:44.852500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa4199-a8ec-45fb-9e11-09f5eb081927 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:44.852802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa4199-a8ec-45fb-9e11-09f5eb081927 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:44.858907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa4199-a8ec-45fb-9e11-09f5eb081927 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:44.859726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa4199-a8ec-45fb-9e11-09f5eb081927 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:44.864020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa4199-a8ec-45fb-9e11-09f5eb081927 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:45.063273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 911/1817] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:44 2022] POST /identity/v3/projects => generated 354 bytes in 346 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:45.067043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa4199-a8ec-45fb-9e11-09f5eb081927 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:45.113680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:45.210276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57cdab9-ba90-406a-b776-2f0ab5960365 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:45.482867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa4199-a8ec-45fb-9e11-09f5eb081927 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:45.533562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947896bc-fe0f-473a-9c3f-603d3a19b868 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b433c680bd4bba95065e344c15855b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4432bdb4b8774e7da14053179c90ca58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:45.534349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947896bc-fe0f-473a-9c3f-603d3a19b868 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:45.534650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947896bc-fe0f-473a-9c3f-603d3a19b868 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:45.534935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947896bc-fe0f-473a-9c3f-603d3a19b868 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:45.549733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947896bc-fe0f-473a-9c3f-603d3a19b868 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:45.553076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947896bc-fe0f-473a-9c3f-603d3a19b868 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:45.554801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947896bc-fe0f-473a-9c3f-603d3a19b868 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:45.643438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:45.644537 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 912/1818] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 581 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:45.648528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:45.649170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:45.649483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] PATH_INFO: `/v3/projects/bcc9dd4c7a0949c98eae0de0ec1d8bfe` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:45.659174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc9dd4c7a0949c98eae0de0ec1d8bfe)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:45.675588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:45.750375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947896bc-fe0f-473a-9c3f-603d3a19b868 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:45.779450 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947896bc-fe0f-473a-9c3f-603d3a19b868 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:45.792995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76b433c680bd4bba95065e344c15855b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4432bdb4b8774e7da14053179c90ca58', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:45.794632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:45.795108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:45.795700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] PATH_INFO: `/v3/projects/af9e0d193b7b47e3b46d37d0537a2c99` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:45.799226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Authorizing `identity:delete_project(project_id=af9e0d193b7b47e3b46d37d0537a2c99)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:45.802904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:45.914379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc9dd4c7a0949c98eae0de0ec1d8bfe', 'request_id': 'req-86e80638-1dab-42d7-bb61-8937f5b8cdee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:45.962655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc9dd4c7a0949c98eae0de0ec1d8bfe', 'request_id': 'req-86e80638-1dab-42d7-bb61-8937f5b8cdee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:45.981224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9e0d193b7b47e3b46d37d0537a2c99', 'request_id': 'req-f92c2698-addc-4bc7-8bfc-c29336d33e82'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:45.995478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9e0d193b7b47e3b46d37d0537a2c99', 'request_id': 'req-f92c2698-addc-4bc7-8bfc-c29336d33e82'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:45.998791 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc9dd4c7a0949c98eae0de0ec1d8bfe', 'request_id': 'req-86e80638-1dab-42d7-bb61-8937f5b8cdee'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:46.044002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] The token cache is being invalidate because project bcc9dd4c7a0949c98eae0de0ec1d8bfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:46.044665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:46.046299 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-86e80638-1dab-42d7-bb61-8937f5b8cdee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:46.050416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9e0d193b7b47e3b46d37d0537a2c99', 'request_id': 'req-f92c2698-addc-4bc7-8bfc-c29336d33e82'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:46.060217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 907/1819] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:44 2022] DELETE /identity/v3/projects/bcc9dd4c7a0949c98eae0de0ec1d8bfe => generated 0 bytes in 1345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:46.067412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc75bef-1c99-4bd4-b900-ea6b7bd8096f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:46.068230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc75bef-1c99-4bd4-b900-ea6b7bd8096f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:46.068805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc75bef-1c99-4bd4-b900-ea6b7bd8096f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:46.094724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] The token cache is being invalidate because project af9e0d193b7b47e3b46d37d0537a2c99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:46.096038 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:46.097297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f92c2698-addc-4bc7-8bfc-c29336d33e82 tempest-VolumeQuotaClassesTest-474671129 tempest-VolumeQuotaClassesTest-474671129-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:46.100452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 913/1820] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:45 2022] DELETE /identity/v3/projects/af9e0d193b7b47e3b46d37d0537a2c99 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:46.135135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0cc75bef-1c99-4bd4-b900-ea6b7bd8096f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:46.145404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0cc75bef-1c99-4bd4-b900-ea6b7bd8096f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:46.249966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:46.348674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 908/1821] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:46.351238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba91cfc-1262-465b-9fb3-ebbf2bd0427c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:46.351536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba91cfc-1262-465b-9fb3-ebbf2bd0427c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:46.351924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba91cfc-1262-465b-9fb3-ebbf2bd0427c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:46.425267 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-eba91cfc-1262-465b-9fb3-ebbf2bd0427c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:46.434645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-eba91cfc-1262-465b-9fb3-ebbf2bd0427c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:46.475185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 914/1822] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:46.478772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc75bef-1c99-4bd4-b900-ea6b7bd8096f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:46.527510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc75bef-1c99-4bd4-b900-ea6b7bd8096f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:46.550351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a40d5-8f99-4697-a04c-fe802378e477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:46.551050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a40d5-8f99-4697-a04c-fe802378e477 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:46.552251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a40d5-8f99-4697-a04c-fe802378e477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:46.552539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a40d5-8f99-4697-a04c-fe802378e477 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:46.553887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a40d5-8f99-4697-a04c-fe802378e477 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:46.553887 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a40d5-8f99-4697-a04c-fe802378e477 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:46.564750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a40d5-8f99-4697-a04c-fe802378e477 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:46.601609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 909/1823] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:46.605938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba91cfc-1262-465b-9fb3-ebbf2bd0427c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:46.621465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba91cfc-1262-465b-9fb3-ebbf2bd0427c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:46.640481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2256936-d03e-4218-9806-2f6f1d32d4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:46.641329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2256936-d03e-4218-9806-2f6f1d32d4cd admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:46.641778 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2256936-d03e-4218-9806-2f6f1d32d4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:46.643512 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2256936-d03e-4218-9806-2f6f1d32d4cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:46.647946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2256936-d03e-4218-9806-2f6f1d32d4cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:46.656030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2256936-d03e-4218-9806-2f6f1d32d4cd admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:46.730459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 915/1824] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:46 2022] POST /identity/v3/projects => generated 365 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:46.731592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a40d5-8f99-4697-a04c-fe802378e477 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:46.755725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a40d5-8f99-4697-a04c-fe802378e477 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:46.787017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2256936-d03e-4218-9806-2f6f1d32d4cd admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:46.789306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e2256936-d03e-4218-9806-2f6f1d32d4cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:46.995742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:47.007928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe71a9-836d-413a-9f51-e8939d5f0d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:47.008678 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe71a9-836d-413a-9f51-e8939d5f0d3a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.008975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe71a9-836d-413a-9f51-e8939d5f0d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.009259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fe71a9-836d-413a-9f51-e8939d5f0d3a admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.018894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe71a9-836d-413a-9f51-e8939d5f0d3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:47.025981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fe71a9-836d-413a-9f51-e8939d5f0d3a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:47.033874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2256936-d03e-4218-9806-2f6f1d32d4cd admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:47.063269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:47.065867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.066177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.066447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] PATH_INFO: `/v3/users/ba3f064fa2894265b1f9072e717b550d` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.071762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba3f064fa2894265b1f9072e717b550d)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:47.079228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:47.103697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 910/1825] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:46 2022] POST /identity/v3/users => generated 283 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:47.109419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe71a9-836d-413a-9f51-e8939d5f0d3a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:47.123523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fe71a9-836d-413a-9f51-e8939d5f0d3a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:47.139776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0c846-f533-405c-9ba7-62fb8d07ced5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:47.141203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0c846-f533-405c-9ba7-62fb8d07ced5 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.142483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0c846-f533-405c-9ba7-62fb8d07ced5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.143238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0c846-f533-405c-9ba7-62fb8d07ced5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.144372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0c846-f533-405c-9ba7-62fb8d07ced5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:47.147177 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0c846-f533-405c-9ba7-62fb8d07ced5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:47.184237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 911/1826] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:27:47.187039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0c846-f533-405c-9ba7-62fb8d07ced5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:47.223203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0c846-f533-405c-9ba7-62fb8d07ced5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:47.224138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba3f064fa2894265b1f9072e717b550d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:47.251016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fe8c5-72c8-48a9-b168-f92a2e2ddc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:47.254669 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fe8c5-72c8-48a9-b168-f92a2e2ddc0e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.255056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fe8c5-72c8-48a9-b168-f92a2e2ddc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.255650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6fe8c5-72c8-48a9-b168-f92a2e2ddc0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.257137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba3f064fa2894265b1f9072e717b550d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:47.257574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fe8c5-72c8-48a9-b168-f92a2e2ddc0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:47.258964 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba3f064fa2894265b1f9072e717b550d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:47.261840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6fe8c5-72c8-48a9-b168-f92a2e2ddc0e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:47.281131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba3f064fa2894265b1f9072e717b550d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:47.292407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 912/1827] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:47 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:47.299945 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fe8c5-72c8-48a9-b168-f92a2e2ddc0e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:47.307529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da3e8803-b544-4170-8079-3c4384eb1794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba3f064fa2894265b1f9072e717b550d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:47.322452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 916/1828] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:46 2022] DELETE /identity/v3/users/ba3f064fa2894265b1f9072e717b550d => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:47.325535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88c301-c5ec-4408-a298-141c3bd6f65e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.325535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88c301-c5ec-4408-a298-141c3bd6f65e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.325765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a88c301-c5ec-4408-a298-141c3bd6f65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.326774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6fe8c5-72c8-48a9-b168-f92a2e2ddc0e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:47.354963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8a88c301-c5ec-4408-a298-141c3bd6f65e None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:47.354963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8a88c301-c5ec-4408-a298-141c3bd6f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:47.421800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb6832-f5bf-4179-b95d-14e50dc25cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:47.422809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb6832-f5bf-4179-b95d-14e50dc25cb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.424201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb6832-f5bf-4179-b95d-14e50dc25cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.424561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb6832-f5bf-4179-b95d-14e50dc25cb6 admin admin] PATH_INFO: `/v3/projects/8ce702fd920b42c595ee357a8bed9cac/users/a207b724a38f4feab8bc5c3f534b2117/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.451099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb6832-f5bf-4179-b95d-14e50dc25cb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ce702fd920b42c595ee357a8bed9cac, user_id=a207b724a38f4feab8bc5c3f534b2117, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:47.451330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 917/1829] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:47.454691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb6832-f5bf-4179-b95d-14e50dc25cb6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:47.462297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88c301-c5ec-4408-a298-141c3bd6f65e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:47.464119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8a88c301-c5ec-4408-a298-141c3bd6f65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:47.483536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 913/1830] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:47 2022] PUT /identity/v3/projects/8ce702fd920b42c595ee357a8bed9cac/users/a207b724a38f4feab8bc5c3f534b2117/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:47.500528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49649-e8e9-4f0f-a2a3-27face0fc5cd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.500831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49649-e8e9-4f0f-a2a3-27face0fc5cd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.501234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b49649-e8e9-4f0f-a2a3-27face0fc5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.540876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-13b49649-e8e9-4f0f-a2a3-27face0fc5cd None None] MFA Rules not processed for user `a207b724a38f4feab8bc5c3f534b2117`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:47.541992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-13b49649-e8e9-4f0f-a2a3-27face0fc5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:47.546528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:47.599839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a88c301-c5ec-4408-a298-141c3bd6f65e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:47.630834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 914/1831] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:47 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:47.632730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579ad6e-e80a-43d3-8e1e-63888494eef1 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.633186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:47.633789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579ad6e-e80a-43d3-8e1e-63888494eef1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.634056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9579ad6e-e80a-43d3-8e1e-63888494eef1 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.635728 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 915/1832] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:27:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:27:47.636452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579ad6e-e80a-43d3-8e1e-63888494eef1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:47.638705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9579ad6e-e80a-43d3-8e1e-63888494eef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:47.673050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e5ea0-c29d-40d5-a811-b57522e94952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:47.674187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e5ea0-c29d-40d5-a811-b57522e94952 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.674473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e5ea0-c29d-40d5-a811-b57522e94952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.674850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e5ea0-c29d-40d5-a811-b57522e94952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.725688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9579ad6e-e80a-43d3-8e1e-63888494eef1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:47.731203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674f4330-01b9-4106-8e2f-5ecc24bc19b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:47.732057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f4330-01b9-4106-8e2f-5ecc24bc19b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.732361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f4330-01b9-4106-8e2f-5ecc24bc19b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.732644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674f4330-01b9-4106-8e2f-5ecc24bc19b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.733896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f4330-01b9-4106-8e2f-5ecc24bc19b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:47.736413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674f4330-01b9-4106-8e2f-5ecc24bc19b8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:47.786295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 916/1833] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:47 2022] POST /identity/v3/projects => generated 369 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:47.788274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e52f0-7b97-4c5d-b605-a9a22cd51eef None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.788575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e52f0-7b97-4c5d-b605-a9a22cd51eef None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.788860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e52f0-7b97-4c5d-b605-a9a22cd51eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.810353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-434e52f0-7b97-4c5d-b605-a9a22cd51eef None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:47.812582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-434e52f0-7b97-4c5d-b605-a9a22cd51eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:47.821606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e5ea0-c29d-40d5-a811-b57522e94952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:47.823292 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e5ea0-c29d-40d5-a811-b57522e94952 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:47.850268 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 918/1834] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:47.851261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e5ea0-c29d-40d5-a811-b57522e94952 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:47.869968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e5ea0-c29d-40d5-a811-b57522e94952 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:47.903141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 917/1835] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:27:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:47.916615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e52f0-7b97-4c5d-b605-a9a22cd51eef None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:47.919869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-434e52f0-7b97-4c5d-b605-a9a22cd51eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:47.948680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e52f0-7b97-4c5d-b605-a9a22cd51eef None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:47.960592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beffc2f-4450-4270-b2c3-7897bf5a71cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:47.961769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beffc2f-4450-4270-b2c3-7897bf5a71cf service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.963162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beffc2f-4450-4270-b2c3-7897bf5a71cf service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.963863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beffc2f-4450-4270-b2c3-7897bf5a71cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:47.981759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beffc2f-4450-4270-b2c3-7897bf5a71cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:47.983769 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beffc2f-4450-4270-b2c3-7897bf5a71cf service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:47.994785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d12bd-ba22-4915-b9ba-039f561251cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:47.996125 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d12bd-ba22-4915-b9ba-039f561251cc admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:47.997192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d12bd-ba22-4915-b9ba-039f561251cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:47.999545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 918/1836] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:27:48.000842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728d12bd-ba22-4915-b9ba-039f561251cc admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.002579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d12bd-ba22-4915-b9ba-039f561251cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.005003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728d12bd-ba22-4915-b9ba-039f561251cc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.043440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 919/1837] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:47 2022] POST /identity/v3/users => generated 291 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:48.047787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beffc2f-4450-4270-b2c3-7897bf5a71cf service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:48.071573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beffc2f-4450-4270-b2c3-7897bf5a71cf service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:48.077520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c06353-8d1f-43e6-bb47-34ba0c228106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:48.078668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c06353-8d1f-43e6-bb47-34ba0c228106 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.078668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c06353-8d1f-43e6-bb47-34ba0c228106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.078930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c06353-8d1f-43e6-bb47-34ba0c228106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.080720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c06353-8d1f-43e6-bb47-34ba0c228106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.082519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c06353-8d1f-43e6-bb47-34ba0c228106 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.085922 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:48.145694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 919/1838] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:48.146667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d12bd-ba22-4915-b9ba-039f561251cc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:48.157413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728d12bd-ba22-4915-b9ba-039f561251cc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:48.181083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732eeb6b-ed78-4f12-9d5f-720accaebfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:48.182069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eeb6b-ed78-4f12-9d5f-720accaebfc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.182420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eeb6b-ed78-4f12-9d5f-720accaebfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.182772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732eeb6b-ed78-4f12-9d5f-720accaebfc8 admin admin] PATH_INFO: `/v3/projects/c28c9e4e4bb14d26885d85047768570a/users/c68c43eab5e148b287634a1ac46b999d/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.203234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732eeb6b-ed78-4f12-9d5f-720accaebfc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28c9e4e4bb14d26885d85047768570a, user_id=c68c43eab5e148b287634a1ac46b999d, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.207294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732eeb6b-ed78-4f12-9d5f-720accaebfc8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.233461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 920/1839] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:48 2022] PUT /identity/v3/projects/c28c9e4e4bb14d26885d85047768570a/users/c68c43eab5e148b287634a1ac46b999d/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:48.234483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c06353-8d1f-43e6-bb47-34ba0c228106 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:48.257526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c06353-8d1f-43e6-bb47-34ba0c228106 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:48.377288 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd1c7b-4309-4c9f-bf88-21366743dce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:48.382826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd1c7b-4309-4c9f-bf88-21366743dce1 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.382826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd1c7b-4309-4c9f-bf88-21366743dce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.383259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd1c7b-4309-4c9f-bf88-21366743dce1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.385093 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd1c7b-4309-4c9f-bf88-21366743dce1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.387172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcd1c7b-4309-4c9f-bf88-21366743dce1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.411462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 920/1840] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:27:48.415315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732eeb6b-ed78-4f12-9d5f-720accaebfc8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:48.429251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732eeb6b-ed78-4f12-9d5f-720accaebfc8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:48.436574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac54e4d-e728-422f-9a66-5bf0b766f4b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:48.440023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac54e4d-e728-422f-9a66-5bf0b766f4b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.440023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac54e4d-e728-422f-9a66-5bf0b766f4b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.440023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac54e4d-e728-422f-9a66-5bf0b766f4b3 admin admin] PATH_INFO: `/v3/projects/c28c9e4e4bb14d26885d85047768570a/users/c68c43eab5e148b287634a1ac46b999d/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.442990 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac54e4d-e728-422f-9a66-5bf0b766f4b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28c9e4e4bb14d26885d85047768570a, user_id=c68c43eab5e148b287634a1ac46b999d, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.447755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac54e4d-e728-422f-9a66-5bf0b766f4b3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.476177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 921/1841] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:48 2022] PUT /identity/v3/projects/c28c9e4e4bb14d26885d85047768570a/users/c68c43eab5e148b287634a1ac46b999d/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:48.476377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd1c7b-4309-4c9f-bf88-21366743dce1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:48.478416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-adcd1c7b-4309-4c9f-bf88-21366743dce1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:48.479785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7b245-1e39-4c89-b60b-7898b4a713e2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.484497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7b245-1e39-4c89-b60b-7898b4a713e2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.485095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7b245-1e39-4c89-b60b-7898b4a713e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.497066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcd1c7b-4309-4c9f-bf88-21366743dce1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:48.528655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3bf7b245-1e39-4c89-b60b-7898b4a713e2 None None] MFA Rules not processed for user `c68c43eab5e148b287634a1ac46b999d`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:48.530520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3bf7b245-1e39-4c89-b60b-7898b4a713e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:48.566908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:48.611964 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:48.613249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.613780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.614237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] PATH_INFO: `/v3/projects/4f652ccd643c4a9db402892e69ff67e2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.618187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f652ccd643c4a9db402892e69ff67e2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.622148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.661720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 922/1842] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:48 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:48.672933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7b245-1e39-4c89-b60b-7898b4a713e2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:48.675197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3bf7b245-1e39-4c89-b60b-7898b4a713e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:48.693595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f652ccd643c4a9db402892e69ff67e2', 'request_id': 'req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.699879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f652ccd643c4a9db402892e69ff67e2', 'request_id': 'req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.710946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f652ccd643c4a9db402892e69ff67e2', 'request_id': 'req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.723675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] The token cache is being invalidate because project 4f652ccd643c4a9db402892e69ff67e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:48.723978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.724767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.730780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 921/1843] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:48 2022] DELETE /identity/v3/projects/4f652ccd643c4a9db402892e69ff67e2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:48.731886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:48.734481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:48.776748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7b245-1e39-4c89-b60b-7898b4a713e2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:48.778845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:48.844075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55a7f2a-3529-4f0f-b57a-74066cb3dcff admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:48.851517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:48.852429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.852721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.853000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] PATH_INFO: `/v3/users/76b433c680bd4bba95065e344c15855b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.854322 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b433c680bd4bba95065e344c15855b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.856307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.879634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ea8b0-2631-465b-a297-57b40e1ac58b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:48.880405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ea8b0-2631-465b-a297-57b40e1ac58b service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:48.880762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ea8b0-2631-465b-a297-57b40e1ac58b service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:48.881654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714ea8b0-2631-465b-a297-57b40e1ac58b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:48.883845 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-714ea8b0-2631-465b-a297-57b40e1ac58b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:48.918757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b433c680bd4bba95065e344c15855b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.930813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b433c680bd4bba95065e344c15855b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.930813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b433c680bd4bba95065e344c15855b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.956086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b433c680bd4bba95065e344c15855b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.965575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ea8b0-2631-465b-a297-57b40e1ac58b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:48.967086 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714ea8b0-2631-465b-a297-57b40e1ac58b service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:48.967208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b433c680bd4bba95065e344c15855b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:48.978119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 922/1844] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:48 2022] DELETE /identity/v3/users/76b433c680bd4bba95065e344c15855b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:48.986799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 923/1845] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:48 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:27:49.039294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:49.040697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:49.113237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520df394-ef78-4c37-9667-6ff08e1f9697 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:49.117656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1070538-6926-483e-9f26-b172eaa88556 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:49.118444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1070538-6926-483e-9f26-b172eaa88556 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:49.118803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1070538-6926-483e-9f26-b172eaa88556 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:49.119145 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1070538-6926-483e-9f26-b172eaa88556 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:49.121153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a1070538-6926-483e-9f26-b172eaa88556 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:49.203489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1070538-6926-483e-9f26-b172eaa88556 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:49.205547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1070538-6926-483e-9f26-b172eaa88556 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:49.209540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:49.237313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 923/1846] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:49 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:49.352590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ea8b0-2631-465b-a297-57b40e1ac58b service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:49.380040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714ea8b0-2631-465b-a297-57b40e1ac58b service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:49.433051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:49.568465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:49.576930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:49.577407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:49.578191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] PATH_INFO: `/v3/projects/4432bdb4b8774e7da14053179c90ca58` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:49.581162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4432bdb4b8774e7da14053179c90ca58)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:49.584790 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:49.696755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4432bdb4b8774e7da14053179c90ca58', 'request_id': 'req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:49.704717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4432bdb4b8774e7da14053179c90ca58', 'request_id': 'req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:49.725528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4432bdb4b8774e7da14053179c90ca58', 'request_id': 'req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:49.748211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] The token cache is being invalidate because project 4432bdb4b8774e7da14053179c90ca58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:49.748510 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:49.749199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f88e6f39-d475-4517-ab8e-0e0dc8f6d060 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:49.754148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 924/1847] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:49 2022] DELETE /identity/v3/projects/4432bdb4b8774e7da14053179c90ca58 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:49.758790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc1f7a-8354-40e4-800b-14c544d78480 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:49.759082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc1f7a-8354-40e4-800b-14c544d78480 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:49.759365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc1f7a-8354-40e4-800b-14c544d78480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:49.792846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-62cc1f7a-8354-40e4-800b-14c544d78480 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:49.794294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-62cc1f7a-8354-40e4-800b-14c544d78480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:49.895668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 924/1848] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:49.905190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5781489-1561-4605-9415-36cf63ab1ffd None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:49.905536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5781489-1561-4605-9415-36cf63ab1ffd None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:49.905844 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5781489-1561-4605-9415-36cf63ab1ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:49.940966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e5781489-1561-4605-9415-36cf63ab1ffd None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:49.942354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e5781489-1561-4605-9415-36cf63ab1ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:49.957521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 925/1849] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:49.962269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc1f7a-8354-40e4-800b-14c544d78480 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:49.984141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc1f7a-8354-40e4-800b-14c544d78480 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:49.999115 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adc9c5-03a6-4316-8f9d-5a8bea571106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.000008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adc9c5-03a6-4316-8f9d-5a8bea571106 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.000576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adc9c5-03a6-4316-8f9d-5a8bea571106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.001003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32adc9c5-03a6-4316-8f9d-5a8bea571106 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.002290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adc9c5-03a6-4316-8f9d-5a8bea571106 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:50.002898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adc9c5-03a6-4316-8f9d-5a8bea571106 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:50.006537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32adc9c5-03a6-4316-8f9d-5a8bea571106 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:50.031751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 925/1850] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:50.039166 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5781489-1561-4605-9415-36cf63ab1ffd None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:50.049748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5781489-1561-4605-9415-36cf63ab1ffd None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:50.058117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef43812-ff3c-4175-9e03-e11d4067210f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.058117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef43812-ff3c-4175-9e03-e11d4067210f admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.058117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef43812-ff3c-4175-9e03-e11d4067210f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.058117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef43812-ff3c-4175-9e03-e11d4067210f admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.058525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef43812-ff3c-4175-9e03-e11d4067210f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:50.059044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef43812-ff3c-4175-9e03-e11d4067210f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:50.094991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 926/1851] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:50 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:27:50.100281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adc9c5-03a6-4316-8f9d-5a8bea571106 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:50.110578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32adc9c5-03a6-4316-8f9d-5a8bea571106 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:50.176591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479bab4-9d10-46e6-9c40-f0e76056d3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.177393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479bab4-9d10-46e6-9c40-f0e76056d3b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.177733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479bab4-9d10-46e6-9c40-f0e76056d3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.178110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479bab4-9d10-46e6-9c40-f0e76056d3b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.178926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479bab4-9d10-46e6-9c40-f0e76056d3b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:50.181124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479bab4-9d10-46e6-9c40-f0e76056d3b8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:50.204007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 926/1852] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:50 2022] POST /identity/v3/users => generated 296 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:50.208987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef43812-ff3c-4175-9e03-e11d4067210f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:50.219562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef43812-ff3c-4175-9e03-e11d4067210f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:50.226793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208bcf5-ad70-4c93-ae82-452ca026a7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.227490 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208bcf5-ad70-4c93-ae82-452ca026a7f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.227774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208bcf5-ad70-4c93-ae82-452ca026a7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.228049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b208bcf5-ad70-4c93-ae82-452ca026a7f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.228799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208bcf5-ad70-4c93-ae82-452ca026a7f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:50.231275 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b208bcf5-ad70-4c93-ae82-452ca026a7f8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:50.246557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 927/1853] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:50.258872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479bab4-9d10-46e6-9c40-f0e76056d3b8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:50.271408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479bab4-9d10-46e6-9c40-f0e76056d3b8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:50.276440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588acdba-2e2f-4c5f-bb9c-92d37663043f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.277174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588acdba-2e2f-4c5f-bb9c-92d37663043f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.277485 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588acdba-2e2f-4c5f-bb9c-92d37663043f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.277772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588acdba-2e2f-4c5f-bb9c-92d37663043f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.278570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588acdba-2e2f-4c5f-bb9c-92d37663043f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:50.280191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588acdba-2e2f-4c5f-bb9c-92d37663043f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:50.291624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 927/1854] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:50.295876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208bcf5-ad70-4c93-ae82-452ca026a7f8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:50.312859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b208bcf5-ad70-4c93-ae82-452ca026a7f8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:50.319908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d05f8b-0fae-414b-9605-41d6ba1770e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.321053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d05f8b-0fae-414b-9605-41d6ba1770e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.321701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d05f8b-0fae-414b-9605-41d6ba1770e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.322218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d05f8b-0fae-414b-9605-41d6ba1770e0 admin admin] PATH_INFO: `/v3/projects/38e77fa803fc47748df06f876bfac3bc/users/2ef1dc9f1baa48fd81732bba4b2cfc92/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.342353 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d05f8b-0fae-414b-9605-41d6ba1770e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e77fa803fc47748df06f876bfac3bc, user_id=2ef1dc9f1baa48fd81732bba4b2cfc92, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:50.346321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d05f8b-0fae-414b-9605-41d6ba1770e0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:50.362117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 928/1855] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:50 2022] PUT /identity/v3/projects/38e77fa803fc47748df06f876bfac3bc/users/2ef1dc9f1baa48fd81732bba4b2cfc92/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:50.372793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb502bd2-69dc-493c-9205-56d3880426a1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.373084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb502bd2-69dc-493c-9205-56d3880426a1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.373363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb502bd2-69dc-493c-9205-56d3880426a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.401278 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fb502bd2-69dc-493c-9205-56d3880426a1 None None] MFA Rules not processed for user `2ef1dc9f1baa48fd81732bba4b2cfc92`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:50.402538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fb502bd2-69dc-493c-9205-56d3880426a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:50.485052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:50.500077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 928/1856] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:50.506637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d05f8b-0fae-414b-9605-41d6ba1770e0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:50.563307 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:50.589652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d05f8b-0fae-414b-9605-41d6ba1770e0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:50.598354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc9ea6-4f75-4803-a8b3-d07b15346992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.599212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc9ea6-4f75-4803-a8b3-d07b15346992 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.600105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc9ea6-4f75-4803-a8b3-d07b15346992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.600385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc9ea6-4f75-4803-a8b3-d07b15346992 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.601126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc9ea6-4f75-4803-a8b3-d07b15346992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:50.603892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fc9ea6-4f75-4803-a8b3-d07b15346992 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:50.636345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 929/1857] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:50 2022] POST /identity/v3/projects => generated 389 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:50.640346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb502bd2-69dc-493c-9205-56d3880426a1 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:50.668844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb502bd2-69dc-493c-9205-56d3880426a1 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:50.797693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83299a-6ffc-48c9-9afb-953aa3f83449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.802883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83299a-6ffc-48c9-9afb-953aa3f83449 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.803176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83299a-6ffc-48c9-9afb-953aa3f83449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.803483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb83299a-6ffc-48c9-9afb-953aa3f83449 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.804534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83299a-6ffc-48c9-9afb-953aa3f83449 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:50.808909 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb83299a-6ffc-48c9-9afb-953aa3f83449 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:50.872902 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 929/1858] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:50 2022] POST /identity/v3/users => generated 301 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:27:50.873021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc9ea6-4f75-4803-a8b3-d07b15346992 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:50.884871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fc9ea6-4f75-4803-a8b3-d07b15346992 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:50.900362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6177c6-6358-4402-b460-a900097dda63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.901656 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6177c6-6358-4402-b460-a900097dda63 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.902192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6177c6-6358-4402-b460-a900097dda63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.902611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6177c6-6358-4402-b460-a900097dda63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:50.904520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6177c6-6358-4402-b460-a900097dda63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:50.908682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6177c6-6358-4402-b460-a900097dda63 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:50.935670 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 930/1859] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:50.937163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83299a-6ffc-48c9-9afb-953aa3f83449 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:50.953020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb83299a-6ffc-48c9-9afb-953aa3f83449 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:50.965858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c901647-ffd8-4faa-96cc-d4e7e61dabfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:50.966868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c901647-ffd8-4faa-96cc-d4e7e61dabfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:50.967220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c901647-ffd8-4faa-96cc-d4e7e61dabfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:50.967615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c901647-ffd8-4faa-96cc-d4e7e61dabfa admin admin] PATH_INFO: `/v3/projects/e0e353ce87654071b5a53c6d90719e75/users/84b4b6f418344318a1cda723bcc65235/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.014111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c901647-ffd8-4faa-96cc-d4e7e61dabfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e353ce87654071b5a53c6d90719e75, user_id=84b4b6f418344318a1cda723bcc65235, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.017071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c901647-ffd8-4faa-96cc-d4e7e61dabfa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.043894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 930/1860] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:50 2022] PUT /identity/v3/projects/e0e353ce87654071b5a53c6d90719e75/users/84b4b6f418344318a1cda723bcc65235/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:51.048958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6177c6-6358-4402-b460-a900097dda63 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.089415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6177c6-6358-4402-b460-a900097dda63 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.215370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92207712-ab5a-438e-9236-390c4e8a313b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.217704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92207712-ab5a-438e-9236-390c4e8a313b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.218055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92207712-ab5a-438e-9236-390c4e8a313b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.218403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92207712-ab5a-438e-9236-390c4e8a313b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.219684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92207712-ab5a-438e-9236-390c4e8a313b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.223149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92207712-ab5a-438e-9236-390c4e8a313b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.256403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 931/1861] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:51.260760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c901647-ffd8-4faa-96cc-d4e7e61dabfa admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.285776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c901647-ffd8-4faa-96cc-d4e7e61dabfa admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.292845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e4a81-c2d9-4bde-8943-4057c57daaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.294349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e4a81-c2d9-4bde-8943-4057c57daaab admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.294921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e4a81-c2d9-4bde-8943-4057c57daaab admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.295291 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e4a81-c2d9-4bde-8943-4057c57daaab admin admin] PATH_INFO: `/v3/projects/e0e353ce87654071b5a53c6d90719e75/users/84b4b6f418344318a1cda723bcc65235/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.299656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e4a81-c2d9-4bde-8943-4057c57daaab admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0e353ce87654071b5a53c6d90719e75, user_id=84b4b6f418344318a1cda723bcc65235, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.303886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2e4a81-c2d9-4bde-8943-4057c57daaab admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.318453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 931/1862] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:51 2022] PUT /identity/v3/projects/e0e353ce87654071b5a53c6d90719e75/users/84b4b6f418344318a1cda723bcc65235/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:51.331077 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622e206-db44-4441-aa77-2efe43e9bacf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.331632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622e206-db44-4441-aa77-2efe43e9bacf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.331912 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4622e206-db44-4441-aa77-2efe43e9bacf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.370606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4622e206-db44-4441-aa77-2efe43e9bacf None None] MFA Rules not processed for user `84b4b6f418344318a1cda723bcc65235`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:51.371862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4622e206-db44-4441-aa77-2efe43e9bacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:51.485460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 932/1863] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:51.506633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e4a81-c2d9-4bde-8943-4057c57daaab admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.508983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ae2e4a81-c2d9-4bde-8943-4057c57daaab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:51.581252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e4a81-c2d9-4bde-8943-4057c57daaab admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.584907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c374704-a563-414a-9fd8-ae5a3b220a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.585603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c374704-a563-414a-9fd8-ae5a3b220a34 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.585892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c374704-a563-414a-9fd8-ae5a3b220a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.586174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c374704-a563-414a-9fd8-ae5a3b220a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.596011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c374704-a563-414a-9fd8-ae5a3b220a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.597490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c374704-a563-414a-9fd8-ae5a3b220a34 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.607148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 932/1864] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:51.659592 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622e206-db44-4441-aa77-2efe43e9bacf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:51.660982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4622e206-db44-4441-aa77-2efe43e9bacf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:51.675201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4622e206-db44-4441-aa77-2efe43e9bacf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:51.678898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaef288-ed7c-4753-8d44-888f84ee68b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:51.679568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef288-ed7c-4753-8d44-888f84ee68b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:51.679886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef288-ed7c-4753-8d44-888f84ee68b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:51.680168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaef288-ed7c-4753-8d44-888f84ee68b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:51.725554 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:51.743939 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaef288-ed7c-4753-8d44-888f84ee68b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:51.745874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaef288-ed7c-4753-8d44-888f84ee68b2 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:51.757132 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 933/1865] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:53.340613 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c374704-a563-414a-9fd8-ae5a3b220a34 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:53.342223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4c374704-a563-414a-9fd8-ae5a3b220a34 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:53.434031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:53.456863 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c374704-a563-414a-9fd8-ae5a3b220a34 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:53.462190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:53.463335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:53.463852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:53.464220 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] PATH_INFO: `/v3/users/a207b724a38f4feab8bc5c3f534b2117` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:53.465559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a207b724a38f4feab8bc5c3f534b2117)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:53.467588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:53.518805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a207b724a38f4feab8bc5c3f534b2117'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:53.525766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a207b724a38f4feab8bc5c3f534b2117'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:53.526461 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a207b724a38f4feab8bc5c3f534b2117'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:53.539308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a207b724a38f4feab8bc5c3f534b2117'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:53.548075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a207b724a38f4feab8bc5c3f534b2117'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:53.554700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 933/1866] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:53 2022] DELETE /identity/v3/users/a207b724a38f4feab8bc5c3f534b2117 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:53.559339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec551872-8584-4b5c-a6e3-e1dcaba88556 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:53.559871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec551872-8584-4b5c-a6e3-e1dcaba88556 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:53.559956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec551872-8584-4b5c-a6e3-e1dcaba88556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:53.579901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ec551872-8584-4b5c-a6e3-e1dcaba88556 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:53.581438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ec551872-8584-4b5c-a6e3-e1dcaba88556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:53.651335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 934/1867] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:53.661124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:53.662609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:53.720217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd8223-5907-4e30-a84b-ba2e921ea255 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:53.723867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b69fd8-c6da-4fbf-aac8-59154b2e35e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:53.724523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b69fd8-c6da-4fbf-aac8-59154b2e35e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:53.724813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b69fd8-c6da-4fbf-aac8-59154b2e35e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:53.725096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b69fd8-c6da-4fbf-aac8-59154b2e35e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:53.734355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b69fd8-c6da-4fbf-aac8-59154b2e35e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:53.736381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b69fd8-c6da-4fbf-aac8-59154b2e35e4 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:53.746232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 934/1868] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:54.046565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec551872-8584-4b5c-a6e3-e1dcaba88556 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:54.049221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ec551872-8584-4b5c-a6e3-e1dcaba88556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:54.066667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec551872-8584-4b5c-a6e3-e1dcaba88556 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:54.073212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:54.074748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:54.074748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:54.075328 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] PATH_INFO: `/v3/projects/8ce702fd920b42c595ee357a8bed9cac` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:54.076853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ce702fd920b42c595ee357a8bed9cac)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:54.079087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:54.173671 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ce702fd920b42c595ee357a8bed9cac', 'request_id': 'req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.185581 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ce702fd920b42c595ee357a8bed9cac', 'request_id': 'req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.205231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ce702fd920b42c595ee357a8bed9cac', 'request_id': 'req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.226955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] The token cache is being invalidate because project 8ce702fd920b42c595ee357a8bed9cac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:54.227465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.228327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.233319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 935/1869] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:54 2022] DELETE /identity/v3/projects/8ce702fd920b42c595ee357a8bed9cac => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:54.244316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b69fd8-c6da-4fbf-aac8-59154b2e35e4 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:54.266311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b69fd8-c6da-4fbf-aac8-59154b2e35e4 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:54.357866 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:54.359297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:54.359679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:54.360060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] PATH_INFO: `/v3/users/c68c43eab5e148b287634a1ac46b999d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:54.361838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c68c43eab5e148b287634a1ac46b999d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:54.363955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:54.383484 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:54.431421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c68c43eab5e148b287634a1ac46b999d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.444966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c68c43eab5e148b287634a1ac46b999d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.445822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c68c43eab5e148b287634a1ac46b999d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.465254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c68c43eab5e148b287634a1ac46b999d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.488331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-819641d8-825a-453b-8946-877b004c6d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c68c43eab5e148b287634a1ac46b999d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:54.496585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 935/1870] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:27:54 2022] DELETE /identity/v3/users/c68c43eab5e148b287634a1ac46b999d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:54.870644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:54.872427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:54.888862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a44062-c9e4-4dea-8a2d-1ab32d28ed90 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:54.955981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:54.956856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:54.957139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:54.957446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] PATH_INFO: `/v3/projects/c28c9e4e4bb14d26885d85047768570a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:54.958635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=c28c9e4e4bb14d26885d85047768570a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:54.960571 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:54.971414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:55.022130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c28c9e4e4bb14d26885d85047768570a', 'request_id': 'req-3529d562-fcf7-4aa4-882d-f6de497aacaa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:55.029850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c28c9e4e4bb14d26885d85047768570a', 'request_id': 'req-3529d562-fcf7-4aa4-882d-f6de497aacaa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:55.041904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c28c9e4e4bb14d26885d85047768570a', 'request_id': 'req-3529d562-fcf7-4aa4-882d-f6de497aacaa'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:55.054090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] The token cache is being invalidate because project c28c9e4e4bb14d26885d85047768570a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:27:55.054374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:55.055017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3529d562-fcf7-4aa4-882d-f6de497aacaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:27:55.056556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 936/1871] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:27:54 2022] DELETE /identity/v3/projects/c28c9e4e4bb14d26885d85047768570a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:55.061516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2d5cb-dd22-4e5d-9225-5855c6a2eb8a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.061818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2d5cb-dd22-4e5d-9225-5855c6a2eb8a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.062104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2d5cb-dd22-4e5d-9225-5855c6a2eb8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.083117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8dd2d5cb-dd22-4e5d-9225-5855c6a2eb8a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:55.084546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8dd2d5cb-dd22-4e5d-9225-5855c6a2eb8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:55.156126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 936/1872] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:55.164483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bb2a0-fbf4-44f2-baed-d25cc5ad7205 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.164774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bb2a0-fbf4-44f2-baed-d25cc5ad7205 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.165055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630bb2a0-fbf4-44f2-baed-d25cc5ad7205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.183105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-630bb2a0-fbf4-44f2-baed-d25cc5ad7205 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:55.184329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-630bb2a0-fbf4-44f2-baed-d25cc5ad7205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:55.193701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 937/1873] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:55.197958 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2d5cb-dd22-4e5d-9225-5855c6a2eb8a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.209819 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2d5cb-dd22-4e5d-9225-5855c6a2eb8a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.215846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15297d-365c-4559-ad19-ed85e98f0dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.216481 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15297d-365c-4559-ad19-ed85e98f0dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.216768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15297d-365c-4559-ad19-ed85e98f0dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.217047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af15297d-365c-4559-ad19-ed85e98f0dcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.217828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15297d-365c-4559-ad19-ed85e98f0dcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:27:55.218472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15297d-365c-4559-ad19-ed85e98f0dcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.221263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af15297d-365c-4559-ad19-ed85e98f0dcd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.236179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 937/1874] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:27:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:27:55.240162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bb2a0-fbf4-44f2-baed-d25cc5ad7205 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.251632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630bb2a0-fbf4-44f2-baed-d25cc5ad7205 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.278686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bde7b1-c10e-4bf5-a4d7-7733e5eab066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.279806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bde7b1-c10e-4bf5-a4d7-7733e5eab066 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.280449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bde7b1-c10e-4bf5-a4d7-7733e5eab066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.280931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bde7b1-c10e-4bf5-a4d7-7733e5eab066 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.284138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bde7b1-c10e-4bf5-a4d7-7733e5eab066 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.297146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bde7b1-c10e-4bf5-a4d7-7733e5eab066 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.333439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 938/1875] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/projects => generated 401 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:55.335174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15297d-365c-4559-ad19-ed85e98f0dcd admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.346409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af15297d-365c-4559-ad19-ed85e98f0dcd admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.430167 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9b159-fb41-4fd2-9f82-39e388cccadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.431678 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9b159-fb41-4fd2-9f82-39e388cccadc admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.432126 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9b159-fb41-4fd2-9f82-39e388cccadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.432434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f9b159-fb41-4fd2-9f82-39e388cccadc admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.433132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9b159-fb41-4fd2-9f82-39e388cccadc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.435203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f9b159-fb41-4fd2-9f82-39e388cccadc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.468817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 938/1876] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/users => generated 301 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:55.473812 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bde7b1-c10e-4bf5-a4d7-7733e5eab066 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.485192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bde7b1-c10e-4bf5-a4d7-7733e5eab066 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.490465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b6f96-185e-4f6e-8139-ee027ef12e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.491225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b6f96-185e-4f6e-8139-ee027ef12e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.491584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b6f96-185e-4f6e-8139-ee027ef12e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.491935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b6f96-185e-4f6e-8139-ee027ef12e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.492962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b6f96-185e-4f6e-8139-ee027ef12e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.495636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1b6f96-185e-4f6e-8139-ee027ef12e10 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.512376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 939/1877] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:55.521565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9b159-fb41-4fd2-9f82-39e388cccadc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.532552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f9b159-fb41-4fd2-9f82-39e388cccadc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.537539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2dfb8c-6095-465f-ac4c-7704b8fc5a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.538217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dfb8c-6095-465f-ac4c-7704b8fc5a92 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.538501 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dfb8c-6095-465f-ac4c-7704b8fc5a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.538773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2dfb8c-6095-465f-ac4c-7704b8fc5a92 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.539498 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dfb8c-6095-465f-ac4c-7704b8fc5a92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.541090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2dfb8c-6095-465f-ac4c-7704b8fc5a92 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.542682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:55.555406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 939/1878] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:55.559450 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b6f96-185e-4f6e-8139-ee027ef12e10 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.572394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1b6f96-185e-4f6e-8139-ee027ef12e10 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.577277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f455fb-1db7-4a25-a51f-6893644b1637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.578604 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f455fb-1db7-4a25-a51f-6893644b1637 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.578830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f455fb-1db7-4a25-a51f-6893644b1637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.579238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f455fb-1db7-4a25-a51f-6893644b1637 admin admin] PATH_INFO: `/v3/projects/d58f3632e14b4c7f85d0c8585e38ea2d/users/c5dde183f1da4c72863952580abea082/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.600321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f455fb-1db7-4a25-a51f-6893644b1637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58f3632e14b4c7f85d0c8585e38ea2d, user_id=c5dde183f1da4c72863952580abea082, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.603413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f455fb-1db7-4a25-a51f-6893644b1637 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.622896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 940/1879] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:55 2022] PUT /identity/v3/projects/d58f3632e14b4c7f85d0c8585e38ea2d/users/c5dde183f1da4c72863952580abea082/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:27:55.635285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22faadf-e2c2-4032-a531-d0d9a50c89c5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.635575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22faadf-e2c2-4032-a531-d0d9a50c89c5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.635860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22faadf-e2c2-4032-a531-d0d9a50c89c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.663233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c22faadf-e2c2-4032-a531-d0d9a50c89c5 None None] MFA Rules not processed for user `c5dde183f1da4c72863952580abea082`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:55.664561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c22faadf-e2c2-4032-a531-d0d9a50c89c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:55.735178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 940/1880] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:55.739970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f455fb-1db7-4a25-a51f-6893644b1637 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.804821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f455fb-1db7-4a25-a51f-6893644b1637 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.809457 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25232e-dcbe-4e7c-948f-68508bdfb0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.810120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25232e-dcbe-4e7c-948f-68508bdfb0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.810403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25232e-dcbe-4e7c-948f-68508bdfb0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.810680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25232e-dcbe-4e7c-948f-68508bdfb0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.811379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25232e-dcbe-4e7c-948f-68508bdfb0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.814135 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25232e-dcbe-4e7c-948f-68508bdfb0b0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.841056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 941/1881] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/projects => generated 399 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:27:55.845207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22faadf-e2c2-4032-a531-d0d9a50c89c5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.856451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22faadf-e2c2-4032-a531-d0d9a50c89c5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.933701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180ee7f-e0b8-4a4f-a789-ff2e391308ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.934447 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180ee7f-e0b8-4a4f-a789-ff2e391308ed admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.934911 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180ee7f-e0b8-4a4f-a789-ff2e391308ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.935265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8180ee7f-e0b8-4a4f-a789-ff2e391308ed admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.935906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180ee7f-e0b8-4a4f-a789-ff2e391308ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.938178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8180ee7f-e0b8-4a4f-a789-ff2e391308ed admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.964512 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 941/1882] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:27:55 2022] POST /identity/v3/users => generated 306 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:27:55.966200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25232e-dcbe-4e7c-948f-68508bdfb0b0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:55.977474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25232e-dcbe-4e7c-948f-68508bdfb0b0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:55.982495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7932ac-56c2-45a1-818e-82520eaa6c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:55.983170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7932ac-56c2-45a1-818e-82520eaa6c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:55.983453 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7932ac-56c2-45a1-818e-82520eaa6c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:55.983726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7932ac-56c2-45a1-818e-82520eaa6c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:55.984467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7932ac-56c2-45a1-818e-82520eaa6c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:55.987198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7932ac-56c2-45a1-818e-82520eaa6c69 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:55.999569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 942/1883] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:27:56.004538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180ee7f-e0b8-4a4f-a789-ff2e391308ed admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:56.014642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8180ee7f-e0b8-4a4f-a789-ff2e391308ed admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.019440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cd2d4-5754-40b5-b6e2-cb069748c155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.020357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cd2d4-5754-40b5-b6e2-cb069748c155 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.020650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cd2d4-5754-40b5-b6e2-cb069748c155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.020926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19cd2d4-5754-40b5-b6e2-cb069748c155 admin admin] PATH_INFO: `/v3/projects/09370d422d414304a07215ecea363571/users/5ddaf6b2722441c28bc14065ec311f1b/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.036228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19cd2d4-5754-40b5-b6e2-cb069748c155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09370d422d414304a07215ecea363571, user_id=5ddaf6b2722441c28bc14065ec311f1b, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.040317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19cd2d4-5754-40b5-b6e2-cb069748c155 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.060029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 942/1884] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:56 2022] PUT /identity/v3/projects/09370d422d414304a07215ecea363571/users/5ddaf6b2722441c28bc14065ec311f1b/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:56.065159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7932ac-56c2-45a1-818e-82520eaa6c69 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:56.081568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7932ac-56c2-45a1-818e-82520eaa6c69 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.087206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:56.160039 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3f2f9-d2b3-4ecc-86d9-1ebd8d76c8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.161152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3f2f9-d2b3-4ecc-86d9-1ebd8d76c8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.161622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3f2f9-d2b3-4ecc-86d9-1ebd8d76c8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.162051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3f2f9-d2b3-4ecc-86d9-1ebd8d76c8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.162947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3f2f9-d2b3-4ecc-86d9-1ebd8d76c8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.164743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3f2f9-d2b3-4ecc-86d9-1ebd8d76c8c8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.179895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 943/1885] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:27:56.185263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cd2d4-5754-40b5-b6e2-cb069748c155 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:56.199193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19cd2d4-5754-40b5-b6e2-cb069748c155 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.208765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dd221-b191-4d01-bdd7-2c4ad22c6c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.208765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dd221-b191-4d01-bdd7-2c4ad22c6c9a admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.209066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dd221-b191-4d01-bdd7-2c4ad22c6c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.209232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93dd221-b191-4d01-bdd7-2c4ad22c6c9a admin admin] PATH_INFO: `/v3/projects/09370d422d414304a07215ecea363571/users/5ddaf6b2722441c28bc14065ec311f1b/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.212138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93dd221-b191-4d01-bdd7-2c4ad22c6c9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=09370d422d414304a07215ecea363571, user_id=5ddaf6b2722441c28bc14065ec311f1b, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.215591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93dd221-b191-4d01-bdd7-2c4ad22c6c9a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.229797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 943/1886] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:27:56 2022] PUT /identity/v3/projects/09370d422d414304a07215ecea363571/users/5ddaf6b2722441c28bc14065ec311f1b/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:27:56.248911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d25ae-10cf-405a-aa43-e2c72e7ca68d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.249211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d25ae-10cf-405a-aa43-e2c72e7ca68d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.249538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2d25ae-10cf-405a-aa43-e2c72e7ca68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.290047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ec2d25ae-10cf-405a-aa43-e2c72e7ca68d None None] MFA Rules not processed for user `5ddaf6b2722441c28bc14065ec311f1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:27:56.292082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ec2d25ae-10cf-405a-aa43-e2c72e7ca68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:56.445294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 944/1887] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:27:56 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:27:56.465880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dd221-b191-4d01-bdd7-2c4ad22c6c9a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:56.469287 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e93dd221-b191-4d01-bdd7-2c4ad22c6c9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:27:56.560199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:27:56.611395 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93dd221-b191-4d01-bdd7-2c4ad22c6c9a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.616941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e5f10-a830-4e38-8835-ee3ac316d4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.617709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e5f10-a830-4e38-8835-ee3ac316d4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.618007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e5f10-a830-4e38-8835-ee3ac316d4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.618290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e5f10-a830-4e38-8835-ee3ac316d4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.652897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e5f10-a830-4e38-8835-ee3ac316d4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.654812 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e5f10-a830-4e38-8835-ee3ac316d4b6 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.695687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 944/1888] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:56 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:27:56.799402 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d25ae-10cf-405a-aa43-e2c72e7ca68d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:27:56.810526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2d25ae-10cf-405a-aa43-e2c72e7ca68d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:27:56.816120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2caabc-0929-48ba-abbe-f52249802e3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:27:56.816807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2caabc-0929-48ba-abbe-f52249802e3e service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:27:56.817094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2caabc-0929-48ba-abbe-f52249802e3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:27:56.817423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2caabc-0929-48ba-abbe-f52249802e3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:27:56.903444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2caabc-0929-48ba-abbe-f52249802e3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:27:56.905316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2caabc-0929-48ba-abbe-f52249802e3e service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:27:56.915347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 945/1889] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:27:56 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:04.717020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e5f10-a830-4e38-8835-ee3ac316d4b6 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:04.811695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e5f10-a830-4e38-8835-ee3ac316d4b6 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:04.821271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:04.825146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:04.825146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:04.825146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] PATH_INFO: `/v3/users/c5dde183f1da4c72863952580abea082` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:04.826572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5dde183f1da4c72863952580abea082)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:04.832950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:04.937101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5dde183f1da4c72863952580abea082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:04.953644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5dde183f1da4c72863952580abea082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:04.955051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5dde183f1da4c72863952580abea082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:04.970871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5dde183f1da4c72863952580abea082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:04.983008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5dde183f1da4c72863952580abea082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:04.996259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 945/1890] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:04 2022] DELETE /identity/v3/users/c5dde183f1da4c72863952580abea082 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:04.998280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9713e-ac0a-45b9-a7bc-fef6707d4c67 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:04.999062 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9713e-ac0a-45b9-a7bc-fef6707d4c67 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:04.999062 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c9713e-ac0a-45b9-a7bc-fef6707d4c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.043948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c6c9713e-ac0a-45b9-a7bc-fef6707d4c67 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:05.049615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c6c9713e-ac0a-45b9-a7bc-fef6707d4c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.117255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:05.253866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 946/1891] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:05.262202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.264974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.283235 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d446d10b-d5de-4b7a-9fb0-2335d78f8f57 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.301622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.303762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.304378 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.304832 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] PATH_INFO: `/v3/users/2ef1dc9f1baa48fd81732bba4b2cfc92` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.306518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ef1dc9f1baa48fd81732bba4b2cfc92)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:05.309040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:05.401959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ef1dc9f1baa48fd81732bba4b2cfc92'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:05.412018 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:05.416298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ef1dc9f1baa48fd81732bba4b2cfc92'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:05.417368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ef1dc9f1baa48fd81732bba4b2cfc92'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:05.449039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ef1dc9f1baa48fd81732bba4b2cfc92'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:05.467929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ef1dc9f1baa48fd81732bba4b2cfc92'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:05.482250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 946/1892] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:05 2022] DELETE /identity/v3/users/2ef1dc9f1baa48fd81732bba4b2cfc92 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:05.488029 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd520b-573c-41a0-bfd2-895838281123 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.488486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd520b-573c-41a0-bfd2-895838281123 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.488850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bd520b-573c-41a0-bfd2-895838281123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.500133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.508425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.553579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c4bd520b-573c-41a0-bfd2-895838281123 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:05.555549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c4bd520b-573c-41a0-bfd2-895838281123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.745834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d21dae0-5b20-4579-a97e-02645775d17a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.759376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.761146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.761505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.761891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.765539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.797043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 947/1893] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:05.799217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd520b-573c-41a0-bfd2-895838281123 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.803240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c4bd520b-573c-41a0-bfd2-895838281123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.827591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bd520b-573c-41a0-bfd2-895838281123 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.842178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.843533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.843997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.844443 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] PATH_INFO: `/v3/users/0372d04ff5604d07a5af4e0ec9c23b1d` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.847236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0372d04ff5604d07a5af4e0ec9c23b1d)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:05.854011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:05.932274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:05.933523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:05.952162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 947/1894] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:05.953146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:05.954973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:05.967902 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397e9b5-c2c1-4dcf-bfa7-4717223941d1 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:05.972298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201d5a66-4197-4d42-80b9-4433dca22ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:05.973017 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5a66-4197-4d42-80b9-4433dca22ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:05.973327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5a66-4197-4d42-80b9-4433dca22ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:05.973849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201d5a66-4197-4d42-80b9-4433dca22ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:05.982392 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0372d04ff5604d07a5af4e0ec9c23b1d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:05.993098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d5a66-4197-4d42-80b9-4433dca22ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:05.994803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201d5a66-4197-4d42-80b9-4433dca22ec2 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.004502 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0372d04ff5604d07a5af4e0ec9c23b1d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:06.005647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0372d04ff5604d07a5af4e0ec9c23b1d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:06.010000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 948/1895] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:06.017236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0372d04ff5604d07a5af4e0ec9c23b1d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:06.046750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0372d04ff5604d07a5af4e0ec9c23b1d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:06.059513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 948/1896] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:05 2022] DELETE /identity/v3/users/0372d04ff5604d07a5af4e0ec9c23b1d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:06.067862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126c02d-9e05-49f1-8474-710471151c17 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.068257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126c02d-9e05-49f1-8474-710471151c17 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.068747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5126c02d-9e05-49f1-8474-710471151c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.097482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5126c02d-9e05-49f1-8474-710471151c17 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:06.100784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5126c02d-9e05-49f1-8474-710471151c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:06.285331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 949/1897] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:06.334748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:06.338631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:06.435762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:06.457940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30de2dc-bc6e-4772-a120-959015d469c0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:06.463931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2066e5f-7757-4170-be9f-cc70eaa2fc32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:06.464862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066e5f-7757-4170-be9f-cc70eaa2fc32 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:06.465295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066e5f-7757-4170-be9f-cc70eaa2fc32 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:06.465765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2066e5f-7757-4170-be9f-cc70eaa2fc32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:06.503001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2066e5f-7757-4170-be9f-cc70eaa2fc32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:06.505025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2066e5f-7757-4170-be9f-cc70eaa2fc32 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:06.536817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 949/1898] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:07.130417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126c02d-9e05-49f1-8474-710471151c17 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.132776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5126c02d-9e05-49f1-8474-710471151c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:07.149858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5126c02d-9e05-49f1-8474-710471151c17 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.162983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.164038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.164707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.164940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] PATH_INFO: `/v3/projects/d58f3632e14b4c7f85d0c8585e38ea2d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.166416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58f3632e14b4c7f85d0c8585e38ea2d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.177674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:07.185387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:07.285127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5306e18-7b3c-4f10-99b2-111b02b09aa9 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.286153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5306e18-7b3c-4f10-99b2-111b02b09aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.287029 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5306e18-7b3c-4f10-99b2-111b02b09aa9 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.288590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 950/1899] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:28:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:28:07.300719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f2d01-9206-42fc-8e12-52b9285d8b13 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.301320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f2d01-9206-42fc-8e12-52b9285d8b13 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.301660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5f2d01-9206-42fc-8e12-52b9285d8b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.312912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58f3632e14b4c7f85d0c8585e38ea2d', 'request_id': 'req-5ce6dbf5-4501-44b7-9e8d-5f0462953080'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.320432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58f3632e14b4c7f85d0c8585e38ea2d', 'request_id': 'req-5ce6dbf5-4501-44b7-9e8d-5f0462953080'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.333826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8a5f2d01-9206-42fc-8e12-52b9285d8b13 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:07.334565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58f3632e14b4c7f85d0c8585e38ea2d', 'request_id': 'req-5ce6dbf5-4501-44b7-9e8d-5f0462953080'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.335089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8a5f2d01-9206-42fc-8e12-52b9285d8b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:07.351127 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] The token cache is being invalidate because project d58f3632e14b4c7f85d0c8585e38ea2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:07.351977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.353878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.357100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 950/1900] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:07 2022] DELETE /identity/v3/projects/d58f3632e14b4c7f85d0c8585e38ea2d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:07.362897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.364991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:07.467938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce6dbf5-4501-44b7-9e8d-5f0462953080 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.474372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.475696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.475894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.476651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] PATH_INFO: `/v3/users/5ddaf6b2722441c28bc14065ec311f1b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.478668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ddaf6b2722441c28bc14065ec311f1b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.481693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:07.504564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 951/1901] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:28:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:07.517292 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f2d01-9206-42fc-8e12-52b9285d8b13 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.519410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8a5f2d01-9206-42fc-8e12-52b9285d8b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:07.555356 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5f2d01-9206-42fc-8e12-52b9285d8b13 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:07.571281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fc7fa-7acd-4c29-a638-accba8b9aeb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:07.572192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fc7fa-7acd-4c29-a638-accba8b9aeb8 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:07.572621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fc7fa-7acd-4c29-a638-accba8b9aeb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:07.573370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760fc7fa-7acd-4c29-a638-accba8b9aeb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:07.580191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ddaf6b2722441c28bc14065ec311f1b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.591813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ddaf6b2722441c28bc14065ec311f1b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.593189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fc7fa-7acd-4c29-a638-accba8b9aeb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:07.593578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ddaf6b2722441c28bc14065ec311f1b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.598379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760fc7fa-7acd-4c29-a638-accba8b9aeb8 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:07.627842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 952/1902] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:28:07.629206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ddaf6b2722441c28bc14065ec311f1b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.643936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ddaf6b2722441c28bc14065ec311f1b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:07.660421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 951/1903] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:07 2022] DELETE /identity/v3/users/5ddaf6b2722441c28bc14065ec311f1b => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:07.948565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fc7fa-7acd-4c29-a638-accba8b9aeb8 service nova] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:07.950402 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-760fc7fa-7acd-4c29-a638-accba8b9aeb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:07.981310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760fc7fa-7acd-4c29-a638-accba8b9aeb8 service nova] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:08.051810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:08.052958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.053163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.053600 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] PATH_INFO: `/v3/projects/38e77fa803fc47748df06f876bfac3bc` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.055060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e77fa803fc47748df06f876bfac3bc)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:08.057209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:08.121181 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e77fa803fc47748df06f876bfac3bc', 'request_id': 'req-7c665dec-333f-4ec4-9c97-b3c590daf814'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.129224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e77fa803fc47748df06f876bfac3bc', 'request_id': 'req-7c665dec-333f-4ec4-9c97-b3c590daf814'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.142518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e77fa803fc47748df06f876bfac3bc', 'request_id': 'req-7c665dec-333f-4ec4-9c97-b3c590daf814'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.145908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:08.168313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] The token cache is being invalidate because project 38e77fa803fc47748df06f876bfac3bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:08.168640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.169415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.172949 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 953/1904] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:07 2022] DELETE /identity/v3/projects/38e77fa803fc47748df06f876bfac3bc => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:08.177535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:08.182833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:08.206918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7b7189-c44f-4483-93ae-94fc23ce2025 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:08.221453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:08.335658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:08.337144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:08.352077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:08.354521 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.355634 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.356076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] PATH_INFO: `/v3/users/84b4b6f418344318a1cda723bcc65235` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.358879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b4b6f418344318a1cda723bcc65235)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:08.361957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:08.425519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c665dec-333f-4ec4-9c97-b3c590daf814 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:08.432393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:08.433680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.433964 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.434240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] PATH_INFO: `/v3/projects/09370d422d414304a07215ecea363571` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.435465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09370d422d414304a07215ecea363571)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:08.439017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:08.448778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b4b6f418344318a1cda723bcc65235'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.457332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b4b6f418344318a1cda723bcc65235'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.458006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b4b6f418344318a1cda723bcc65235'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.473536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b4b6f418344318a1cda723bcc65235'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.483176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b4b6f418344318a1cda723bcc65235'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.493263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 952/1905] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:08 2022] DELETE /identity/v3/users/84b4b6f418344318a1cda723bcc65235 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:08.500346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09370d422d414304a07215ecea363571', 'request_id': 'req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.508175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09370d422d414304a07215ecea363571', 'request_id': 'req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.518820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09370d422d414304a07215ecea363571', 'request_id': 'req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.533098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:08.534920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:08.562613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] The token cache is being invalidate because project 09370d422d414304a07215ecea363571 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:08.563135 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.564280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-b54fc5a6-4ca7-4f80-94ac-4584bf45bb23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.570955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 954/1906] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:08 2022] DELETE /identity/v3/projects/09370d422d414304a07215ecea363571 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:08.576868 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d685fb-48a2-4e21-8819-76bef8cd9a03 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.577560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d685fb-48a2-4e21-8819-76bef8cd9a03 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.578565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d685fb-48a2-4e21-8819-76bef8cd9a03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.621582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c3d685fb-48a2-4e21-8819-76bef8cd9a03 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:08.623754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c3d685fb-48a2-4e21-8819-76bef8cd9a03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:08.655519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e14319-1b76-47d9-b815-773e08d8c40a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:08.664550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:08.666352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.666651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.667800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] PATH_INFO: `/v3/projects/8680e2820f5441daa1fc34b008a96684` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.669282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=8680e2820f5441daa1fc34b008a96684)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:08.673413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:08.699929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 955/1907] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:08.709042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e601538-b680-4e54-95bd-6b479edaa8ce None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.710164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e601538-b680-4e54-95bd-6b479edaa8ce None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.710518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e601538-b680-4e54-95bd-6b479edaa8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.735893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9e601538-b680-4e54-95bd-6b479edaa8ce None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:08.737144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:08.737771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9e601538-b680-4e54-95bd-6b479edaa8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:08.757170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8680e2820f5441daa1fc34b008a96684', 'request_id': 'req-51a8114c-3e4d-4faa-acf5-193e242556ad'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.764772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8680e2820f5441daa1fc34b008a96684', 'request_id': 'req-51a8114c-3e4d-4faa-acf5-193e242556ad'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.766863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:08.775749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8680e2820f5441daa1fc34b008a96684', 'request_id': 'req-51a8114c-3e4d-4faa-acf5-193e242556ad'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.791284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] The token cache is being invalidate because project 8680e2820f5441daa1fc34b008a96684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:08.792186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.794525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51a8114c-3e4d-4faa-acf5-193e242556ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:08.805460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 953/1908] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:08 2022] DELETE /identity/v3/projects/8680e2820f5441daa1fc34b008a96684 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:08.807762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c46dd-6716-47db-8762-58938215b994 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.808165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c46dd-6716-47db-8762-58938215b994 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.808809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00c46dd-6716-47db-8762-58938215b994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.831156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a00c46dd-6716-47db-8762-58938215b994 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:08.832487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a00c46dd-6716-47db-8762-58938215b994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:08.881538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 956/1909] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:08.883572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e601538-b680-4e54-95bd-6b479edaa8ce None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:08.895112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e601538-b680-4e54-95bd-6b479edaa8ce None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:08.900891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d5d06-6130-4e85-962b-3d00563ac266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:08.901993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d5d06-6130-4e85-962b-3d00563ac266 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.903011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d5d06-6130-4e85-962b-3d00563ac266 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.903011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d5d06-6130-4e85-962b-3d00563ac266 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.903389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d5d06-6130-4e85-962b-3d00563ac266 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:08.904565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d5d06-6130-4e85-962b-3d00563ac266 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:08.906282 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d5d06-6130-4e85-962b-3d00563ac266 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:08.923996 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 957/1910] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:08.928642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d5d06-6130-4e85-962b-3d00563ac266 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:08.933117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 954/1911] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:08.941224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d5d06-6130-4e85-962b-3d00563ac266 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:08.944367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736190e6-56dc-4a74-ba41-69a8477d6312 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.944677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736190e6-56dc-4a74-ba41-69a8477d6312 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.944963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736190e6-56dc-4a74-ba41-69a8477d6312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.951233 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76bef3-6c1a-46f3-a822-38ca4801e4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:08.952236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76bef3-6c1a-46f3-a822-38ca4801e4f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:08.952730 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76bef3-6c1a-46f3-a822-38ca4801e4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:08.953498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de76bef3-6c1a-46f3-a822-38ca4801e4f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:08.954482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76bef3-6c1a-46f3-a822-38ca4801e4f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:08.960215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de76bef3-6c1a-46f3-a822-38ca4801e4f1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:08.987251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-736190e6-56dc-4a74-ba41-69a8477d6312 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:08.988662 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-736190e6-56dc-4a74-ba41-69a8477d6312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:08.998954 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 955/1912] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:09.004550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736190e6-56dc-4a74-ba41-69a8477d6312 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.005267 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 958/1913] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:08 2022] POST /identity/v3/projects => generated 365 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:09.014636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76bef3-6c1a-46f3-a822-38ca4801e4f1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.019599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736190e6-56dc-4a74-ba41-69a8477d6312 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.090616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de76bef3-6c1a-46f3-a822-38ca4801e4f1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.095308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.095546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91669fcf-4a90-4a06-877e-eb13039c9c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.095978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.096214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669fcf-4a90-4a06-877e-eb13039c9c61 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.096316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.096503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669fcf-4a90-4a06-877e-eb13039c9c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.096593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.096782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669fcf-4a90-4a06-877e-eb13039c9c61 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.097244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:09.097551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91669fcf-4a90-4a06-877e-eb13039c9c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.097677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.099367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.099473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91669fcf-4a90-4a06-877e-eb13039c9c61 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.114156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 956/1914] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:09.115129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.117035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:09.121396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 959/1915] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/users => generated 283 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:09.125258 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91669fcf-4a90-4a06-877e-eb13039c9c61 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.130001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62360da0-cf00-455f-a639-b1fabf5ccf5b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.134890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.135753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.136038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.136174 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91669fcf-4a90-4a06-877e-eb13039c9c61 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.136312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] PATH_INFO: `/v3/projects/e0e353ce87654071b5a53c6d90719e75` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.137507 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0e353ce87654071b5a53c6d90719e75)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.139439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.140689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b68ed1-174a-46c5-a569-20d8e9fcafe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.142023 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b68ed1-174a-46c5-a569-20d8e9fcafe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.142287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b68ed1-174a-46c5-a569-20d8e9fcafe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.142570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b68ed1-174a-46c5-a569-20d8e9fcafe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.143314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b68ed1-174a-46c5-a569-20d8e9fcafe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.146283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b68ed1-174a-46c5-a569-20d8e9fcafe6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.176690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 960/1916] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:09.176690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b68ed1-174a-46c5-a569-20d8e9fcafe6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.187984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:09.193386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0e353ce87654071b5a53c6d90719e75', 'request_id': 'req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:09.202219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0e353ce87654071b5a53c6d90719e75', 'request_id': 'req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:09.212872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0e353ce87654071b5a53c6d90719e75', 'request_id': 'req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:09.228061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] The token cache is being invalidate because project e0e353ce87654071b5a53c6d90719e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:09.228516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:09.229314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:09.231370 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 957/1917] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:09 2022] DELETE /identity/v3/projects/e0e353ce87654071b5a53c6d90719e75 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:09.232352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.234069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:09.243531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:09.248437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b68ed1-174a-46c5-a569-20d8e9fcafe6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.306761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae56faca-3f52-44a9-afa6-9b1908fc6c1f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.310379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd0a09-7196-41ff-b547-b59471068fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.311093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd0a09-7196-41ff-b547-b59471068fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.311487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd0a09-7196-41ff-b547-b59471068fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.311589 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4092-f555-49cd-a2a0-7b360366a473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.311813 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd0a09-7196-41ff-b547-b59471068fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.312163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4092-f555-49cd-a2a0-7b360366a473 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.312713 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4092-f555-49cd-a2a0-7b360366a473 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.312994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ea4092-f555-49cd-a2a0-7b360366a473 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.313082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd0a09-7196-41ff-b547-b59471068fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.314532 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea4092-f555-49cd-a2a0-7b360366a473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.315705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd0a09-7196-41ff-b547-b59471068fd2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.316455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ea4092-f555-49cd-a2a0-7b360366a473 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.331936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 961/1918] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:09.331936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11f08-fada-4d00-97d1-3754400490df None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.331936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11f08-fada-4d00-97d1-3754400490df None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.331936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec11f08-fada-4d00-97d1-3754400490df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.340358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 958/1919] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/users => generated 288 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:09.341248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4092-f555-49cd-a2a0-7b360366a473 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.343181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-45ea4092-f555-49cd-a2a0-7b360366a473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:09.355976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ea4092-f555-49cd-a2a0-7b360366a473 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.357578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7ec11f08-fada-4d00-97d1-3754400490df None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:09.358783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7ec11f08-fada-4d00-97d1-3754400490df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:09.361545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df073426-a13c-411e-ba38-7d48acec785d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.362251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df073426-a13c-411e-ba38-7d48acec785d admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.362542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df073426-a13c-411e-ba38-7d48acec785d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.362825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df073426-a13c-411e-ba38-7d48acec785d admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.363593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df073426-a13c-411e-ba38-7d48acec785d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.365982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df073426-a13c-411e-ba38-7d48acec785d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.367981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 962/1920] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:09.368930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec11f08-fada-4d00-97d1-3754400490df None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.383785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec11f08-fada-4d00-97d1-3754400490df None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.384876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 959/1921] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:09.386263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27514372-66f4-446b-806b-f7b43a8d8395 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.386559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27514372-66f4-446b-806b-f7b43a8d8395 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.386864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27514372-66f4-446b-806b-f7b43a8d8395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.389204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae31d2b-b4e3-48de-9efa-a9ee519cb3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.389904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae31d2b-b4e3-48de-9efa-a9ee519cb3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.390186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae31d2b-b4e3-48de-9efa-a9ee519cb3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.390462 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae31d2b-b4e3-48de-9efa-a9ee519cb3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.391197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae31d2b-b4e3-48de-9efa-a9ee519cb3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.393891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae31d2b-b4e3-48de-9efa-a9ee519cb3c9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.407757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 963/1922] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:09.407986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-27514372-66f4-446b-806b-f7b43a8d8395 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:09.408095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae31d2b-b4e3-48de-9efa-a9ee519cb3c9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.408185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-27514372-66f4-446b-806b-f7b43a8d8395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:09.415530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae31d2b-b4e3-48de-9efa-a9ee519cb3c9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.419013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 960/1923] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:09.419013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27514372-66f4-446b-806b-f7b43a8d8395 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.420532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52087be3-9050-4c20-bc4e-c4d930dfc33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.421512 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52087be3-9050-4c20-bc4e-c4d930dfc33c admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.421796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52087be3-9050-4c20-bc4e-c4d930dfc33c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.422074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52087be3-9050-4c20-bc4e-c4d930dfc33c admin admin] PATH_INFO: `/v3/projects/1a5fb6af1b20439589ca660c05ba2b62/users/33bf221cf90b48b8ac5297bddba1bbb2/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.426655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27514372-66f4-446b-806b-f7b43a8d8395 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.431699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc878e-db40-414d-bb2b-d4824a5920c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.432367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc878e-db40-414d-bb2b-d4824a5920c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.432653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc878e-db40-414d-bb2b-d4824a5920c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.432933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbc878e-db40-414d-bb2b-d4824a5920c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.433748 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc878e-db40-414d-bb2b-d4824a5920c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.435333 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbc878e-db40-414d-bb2b-d4824a5920c2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.439930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52087be3-9050-4c20-bc4e-c4d930dfc33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a5fb6af1b20439589ca660c05ba2b62, user_id=33bf221cf90b48b8ac5297bddba1bbb2, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.443721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52087be3-9050-4c20-bc4e-c4d930dfc33c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.446831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 961/1924] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:09.447663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc878e-db40-414d-bb2b-d4824a5920c2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.458417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbc878e-db40-414d-bb2b-d4824a5920c2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.460222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 964/1925] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:09 2022] PUT /identity/v3/projects/1a5fb6af1b20439589ca660c05ba2b62/users/33bf221cf90b48b8ac5297bddba1bbb2/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:09.461486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52087be3-9050-4c20-bc4e-c4d930dfc33c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.533231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f92de0-7aaf-4f47-ad6a-bb08fe3c2e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.533231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92de0-7aaf-4f47-ad6a-bb08fe3c2e73 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.533231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92de0-7aaf-4f47-ad6a-bb08fe3c2e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.533231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f92de0-7aaf-4f47-ad6a-bb08fe3c2e73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.533651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92de0-7aaf-4f47-ad6a-bb08fe3c2e73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:09.534111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92de0-7aaf-4f47-ad6a-bb08fe3c2e73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.535919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f92de0-7aaf-4f47-ad6a-bb08fe3c2e73 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.540253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52087be3-9050-4c20-bc4e-c4d930dfc33c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.545261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f684b2-f7eb-4a86-9fef-d2c7ccda5706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.546337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f684b2-f7eb-4a86-9fef-d2c7ccda5706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.546688 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f684b2-f7eb-4a86-9fef-d2c7ccda5706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.547041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f684b2-f7eb-4a86-9fef-d2c7ccda5706 admin admin] PATH_INFO: `/v3/projects/2da864dadf614f15a06a00fded3fd819/users/2d0b3d8a81c34267ac6d4e2a68764c73/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.550914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 962/1926] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:09.552530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac172-faf0-4e82-a87c-d1233c8b21db None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.553102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac172-faf0-4e82-a87c-d1233c8b21db None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.553406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ac172-faf0-4e82-a87c-d1233c8b21db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.572229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f684b2-f7eb-4a86-9fef-d2c7ccda5706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da864dadf614f15a06a00fded3fd819, user_id=2d0b3d8a81c34267ac6d4e2a68764c73, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.575863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f684b2-f7eb-4a86-9fef-d2c7ccda5706 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.584038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8a4ac172-faf0-4e82-a87c-d1233c8b21db None None] MFA Rules not processed for user `33bf221cf90b48b8ac5297bddba1bbb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:09.586212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8a4ac172-faf0-4e82-a87c-d1233c8b21db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:09.599416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 965/1927] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:09 2022] PUT /identity/v3/projects/2da864dadf614f15a06a00fded3fd819/users/2d0b3d8a81c34267ac6d4e2a68764c73/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:09.600979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:09.609189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f684b2-f7eb-4a86-9fef-d2c7ccda5706 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.668523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 963/1928] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:09.670754 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506e228-8293-41c1-abde-a7dad1c291bb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.671230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506e228-8293-41c1-abde-a7dad1c291bb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.671791 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506e228-8293-41c1-abde-a7dad1c291bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.677075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:09.705901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c506e228-8293-41c1-abde-a7dad1c291bb None None] MFA Rules not processed for user `2d0b3d8a81c34267ac6d4e2a68764c73`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:09.707237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c506e228-8293-41c1-abde-a7dad1c291bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:09.708233 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f684b2-f7eb-4a86-9fef-d2c7ccda5706 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.713236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7e546-c437-4457-be7e-b847fb8f6777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.713940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7e546-c437-4457-be7e-b847fb8f6777 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.714242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7e546-c437-4457-be7e-b847fb8f6777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.714507 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7e546-c437-4457-be7e-b847fb8f6777 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.715211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7e546-c437-4457-be7e-b847fb8f6777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.717176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a7e546-c437-4457-be7e-b847fb8f6777 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.759574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 966/1929] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/projects => generated 407 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:09.759574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7e546-c437-4457-be7e-b847fb8f6777 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.787438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 964/1930] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:09.788399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506e228-8293-41c1-abde-a7dad1c291bb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.799315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506e228-8293-41c1-abde-a7dad1c291bb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.820630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7e546-c437-4457-be7e-b847fb8f6777 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.825407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89480403-16a3-4fc8-83e1-f9068f255d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.826079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89480403-16a3-4fc8-83e1-f9068f255d69 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.826373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89480403-16a3-4fc8-83e1-f9068f255d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.826653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89480403-16a3-4fc8-83e1-f9068f255d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.827383 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89480403-16a3-4fc8-83e1-f9068f255d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.829765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89480403-16a3-4fc8-83e1-f9068f255d69 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.857064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 967/1931] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:09.857941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89480403-16a3-4fc8-83e1-f9068f255d69 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.866058 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1211-7ec8-48da-a36c-a383b1568cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.866672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1211-7ec8-48da-a36c-a383b1568cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.866963 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1211-7ec8-48da-a36c-a383b1568cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.867246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79df1211-7ec8-48da-a36c-a383b1568cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.868361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1211-7ec8-48da-a36c-a383b1568cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.870381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79df1211-7ec8-48da-a36c-a383b1568cf3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.898019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 965/1932] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/users => generated 304 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:09.901980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1211-7ec8-48da-a36c-a383b1568cf3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.912918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79df1211-7ec8-48da-a36c-a383b1568cf3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.935197 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89480403-16a3-4fc8-83e1-f9068f255d69 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:09.940504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d519d-f849-4747-b7a6-45ce8c7fc54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.941182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d519d-f849-4747-b7a6-45ce8c7fc54d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.941498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d519d-f849-4747-b7a6-45ce8c7fc54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.941782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d519d-f849-4747-b7a6-45ce8c7fc54d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.942487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d519d-f849-4747-b7a6-45ce8c7fc54d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:09.944423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d519d-f849-4747-b7a6-45ce8c7fc54d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:09.954312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:09.980054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 968/1933] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:09.980054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d519d-f849-4747-b7a6-45ce8c7fc54d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:09.996343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123348-eaec-4cc2-baea-94e8c95f939a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:09.997035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123348-eaec-4cc2-baea-94e8c95f939a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:09.997328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123348-eaec-4cc2-baea-94e8c95f939a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:09.997652 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49123348-eaec-4cc2-baea-94e8c95f939a admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:09.998356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49123348-eaec-4cc2-baea-94e8c95f939a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.000524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49123348-eaec-4cc2-baea-94e8c95f939a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.025425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 966/1934] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:09 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:10.025425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123348-eaec-4cc2-baea-94e8c95f939a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.034372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49123348-eaec-4cc2-baea-94e8c95f939a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.036359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:10.076970 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d519d-f849-4747-b7a6-45ce8c7fc54d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.084675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c338261-8352-4e21-af7c-d89d37f7bb38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.085366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c338261-8352-4e21-af7c-d89d37f7bb38 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.085680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c338261-8352-4e21-af7c-d89d37f7bb38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.085958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c338261-8352-4e21-af7c-d89d37f7bb38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.086729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c338261-8352-4e21-af7c-d89d37f7bb38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.088349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c338261-8352-4e21-af7c-d89d37f7bb38 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.100418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 969/1935] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:09 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:10.101276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c338261-8352-4e21-af7c-d89d37f7bb38 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.113429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c338261-8352-4e21-af7c-d89d37f7bb38 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.118563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15ff2f-95a5-410c-97e8-1028001dec38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.119244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15ff2f-95a5-410c-97e8-1028001dec38 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.119535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15ff2f-95a5-410c-97e8-1028001dec38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.119814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e15ff2f-95a5-410c-97e8-1028001dec38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.120552 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15ff2f-95a5-410c-97e8-1028001dec38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.122124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c55dac-9bb3-48e9-9d49-20f200430cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.122275 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e15ff2f-95a5-410c-97e8-1028001dec38 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.122820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c55dac-9bb3-48e9-9d49-20f200430cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.123099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c55dac-9bb3-48e9-9d49-20f200430cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.123375 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c55dac-9bb3-48e9-9d49-20f200430cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.124077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c55dac-9bb3-48e9-9d49-20f200430cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.126629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c55dac-9bb3-48e9-9d49-20f200430cd0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.134425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 970/1936] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:10.135385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15ff2f-95a5-410c-97e8-1028001dec38 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.146274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e15ff2f-95a5-410c-97e8-1028001dec38 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.151464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc1fb0-e009-4e86-8e1d-32b0dabdfad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.152221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc1fb0-e009-4e86-8e1d-32b0dabdfad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.152576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc1fb0-e009-4e86-8e1d-32b0dabdfad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.153060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 967/1937] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:10 2022] POST /identity/v3/users => generated 293 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:10.153584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc1fb0-e009-4e86-8e1d-32b0dabdfad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.154222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c55dac-9bb3-48e9-9d49-20f200430cd0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.154649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc1fb0-e009-4e86-8e1d-32b0dabdfad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.156483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfc1fb0-e009-4e86-8e1d-32b0dabdfad4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.166992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c55dac-9bb3-48e9-9d49-20f200430cd0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.168248 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 971/1938] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:10.169115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc1fb0-e009-4e86-8e1d-32b0dabdfad4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.183590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b11b-1cb5-4f12-94c9-a43bdf346d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.185949 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b11b-1cb5-4f12-94c9-a43bdf346d6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.186414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b11b-1cb5-4f12-94c9-a43bdf346d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.186839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208b11b-1cb5-4f12-94c9-a43bdf346d6b admin admin] PATH_INFO: `/v3/projects/c72c03b6e3d84feab93c73129e4f720a/users/f54d72eea3684542879eab8ddd86aa90/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.187127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc1fb0-e009-4e86-8e1d-32b0dabdfad4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.195039 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b72ee1-c94b-426a-b8b4-e190307ea86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.195714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b72ee1-c94b-426a-b8b4-e190307ea86c admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.196006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b72ee1-c94b-426a-b8b4-e190307ea86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.196324 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b72ee1-c94b-426a-b8b4-e190307ea86c admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.197049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b72ee1-c94b-426a-b8b4-e190307ea86c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.198907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b72ee1-c94b-426a-b8b4-e190307ea86c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.209116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b11b-1cb5-4f12-94c9-a43bdf346d6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72c03b6e3d84feab93c73129e4f720a, user_id=f54d72eea3684542879eab8ddd86aa90, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.212542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4208b11b-1cb5-4f12-94c9-a43bdf346d6b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.216312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 972/1939] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:10.216312 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b72ee1-c94b-426a-b8b4-e190307ea86c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.232377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 968/1940] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:10 2022] PUT /identity/v3/projects/c72c03b6e3d84feab93c73129e4f720a/users/f54d72eea3684542879eab8ddd86aa90/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:10.233646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b11b-1cb5-4f12-94c9-a43bdf346d6b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.252203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b72ee1-c94b-426a-b8b4-e190307ea86c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.328538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208b11b-1cb5-4f12-94c9-a43bdf346d6b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.333738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baebec4-1f45-41c3-8ff6-b35fb70ca267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.334743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baebec4-1f45-41c3-8ff6-b35fb70ca267 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.334941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baebec4-1f45-41c3-8ff6-b35fb70ca267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.335315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baebec4-1f45-41c3-8ff6-b35fb70ca267 admin admin] PATH_INFO: `/v3/projects/3ae607f184164acebfd8f382a9a64285/users/d74694d70d0a4580b254ab6f598ac082/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.341199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4050555-b658-425e-9140-d3d02cdb2866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.342218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4050555-b658-425e-9140-d3d02cdb2866 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.342504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4050555-b658-425e-9140-d3d02cdb2866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.342785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4050555-b658-425e-9140-d3d02cdb2866 admin admin] PATH_INFO: `/v3/projects/0122b63b80e34f299ffd0afd403dea34/users/38845d328a544efbae036e0f726464be/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.352871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baebec4-1f45-41c3-8ff6-b35fb70ca267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae607f184164acebfd8f382a9a64285, user_id=d74694d70d0a4580b254ab6f598ac082, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.355778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baebec4-1f45-41c3-8ff6-b35fb70ca267 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.359144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4050555-b658-425e-9140-d3d02cdb2866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0122b63b80e34f299ffd0afd403dea34, user_id=38845d328a544efbae036e0f726464be, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.363327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4050555-b658-425e-9140-d3d02cdb2866 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.370778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 969/1941] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:10 2022] PUT /identity/v3/projects/3ae607f184164acebfd8f382a9a64285/users/d74694d70d0a4580b254ab6f598ac082/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:10.371896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baebec4-1f45-41c3-8ff6-b35fb70ca267 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.383284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 973/1942] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:10 2022] PUT /identity/v3/projects/0122b63b80e34f299ffd0afd403dea34/users/38845d328a544efbae036e0f726464be/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:10.384699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4050555-b658-425e-9140-d3d02cdb2866 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.394015 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:10.445957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baebec4-1f45-41c3-8ff6-b35fb70ca267 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.461558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4050555-b658-425e-9140-d3d02cdb2866 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.467557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac390be-fdf1-4297-9bc7-cb66e2ab0d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.468245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac390be-fdf1-4297-9bc7-cb66e2ab0d66 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.468656 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac390be-fdf1-4297-9bc7-cb66e2ab0d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.468942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac390be-fdf1-4297-9bc7-cb66e2ab0d66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.470276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac390be-fdf1-4297-9bc7-cb66e2ab0d66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.471898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac390be-fdf1-4297-9bc7-cb66e2ab0d66 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.483642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 974/1943] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:10.484964 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74d68-4f4e-4941-8e73-19e6df12be2c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.486053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74d68-4f4e-4941-8e73-19e6df12be2c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.486342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a74d68-4f4e-4941-8e73-19e6df12be2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.513246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63bcea-d597-4919-8959-02ca12e6ffb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.513993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63bcea-d597-4919-8959-02ca12e6ffb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.514603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63bcea-d597-4919-8959-02ca12e6ffb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.514603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b63bcea-d597-4919-8959-02ca12e6ffb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.515761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63bcea-d597-4919-8959-02ca12e6ffb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.518331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b63bcea-d597-4919-8959-02ca12e6ffb2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.519550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d5a74d68-4f4e-4941-8e73-19e6df12be2c None None] MFA Rules not processed for user `38845d328a544efbae036e0f726464be`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:10.520800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d5a74d68-4f4e-4941-8e73-19e6df12be2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:10.528134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:10.533105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 970/1944] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:10.533105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63bcea-d597-4919-8959-02ca12e6ffb2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.542813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b63bcea-d597-4919-8959-02ca12e6ffb2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.550157 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72b688-2fdd-4f62-a14a-56ccd98abd99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.550995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b688-2fdd-4f62-a14a-56ccd98abd99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.551282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b688-2fdd-4f62-a14a-56ccd98abd99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.551565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e72b688-2fdd-4f62-a14a-56ccd98abd99 admin admin] PATH_INFO: `/v3/projects/3ae607f184164acebfd8f382a9a64285/users/d74694d70d0a4580b254ab6f598ac082/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.554060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72b688-2fdd-4f62-a14a-56ccd98abd99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae607f184164acebfd8f382a9a64285, user_id=d74694d70d0a4580b254ab6f598ac082, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.556969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e72b688-2fdd-4f62-a14a-56ccd98abd99 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.568836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 971/1945] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:10 2022] PUT /identity/v3/projects/3ae607f184164acebfd8f382a9a64285/users/d74694d70d0a4580b254ab6f598ac082/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:10.578085 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72b688-2fdd-4f62-a14a-56ccd98abd99 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.619395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 975/1946] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:10 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:10.619395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1cd07-8391-429c-ae37-923d01c7bc25 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.619395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1cd07-8391-429c-ae37-923d01c7bc25 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.619395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee1cd07-8391-429c-ae37-923d01c7bc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.650663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e72b688-2fdd-4f62-a14a-56ccd98abd99 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.653133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1ee1cd07-8391-429c-ae37-923d01c7bc25 None None] MFA Rules not processed for user `d74694d70d0a4580b254ab6f598ac082`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:10.654374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1ee1cd07-8391-429c-ae37-923d01c7bc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:10.656203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad022e-851c-4a68-87b6-060d0fae51e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.657353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad022e-851c-4a68-87b6-060d0fae51e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.657984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad022e-851c-4a68-87b6-060d0fae51e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.658063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ad022e-851c-4a68-87b6-060d0fae51e6 admin admin] PATH_INFO: `/v3/projects/c72c03b6e3d84feab93c73129e4f720a/users/f54d72eea3684542879eab8ddd86aa90/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.660360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad022e-851c-4a68-87b6-060d0fae51e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c72c03b6e3d84feab93c73129e4f720a, user_id=f54d72eea3684542879eab8ddd86aa90, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.663477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ad022e-851c-4a68-87b6-060d0fae51e6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.686821 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 972/1947] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:10 2022] PUT /identity/v3/projects/c72c03b6e3d84feab93c73129e4f720a/users/f54d72eea3684542879eab8ddd86aa90/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:10.687930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad022e-851c-4a68-87b6-060d0fae51e6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.762265 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 976/1948] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:10 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:10.764035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d7545-a01f-4c70-a063-09a434080ede None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.764396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d7545-a01f-4c70-a063-09a434080ede None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.764740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d7545-a01f-4c70-a063-09a434080ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.768489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:10.807427 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ad022e-851c-4a68-87b6-060d0fae51e6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.813881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35e67d-e499-46a0-a3f3-d1f0644560f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.816107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35e67d-e499-46a0-a3f3-d1f0644560f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.816746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35e67d-e499-46a0-a3f3-d1f0644560f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.817504 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb35e67d-e499-46a0-a3f3-d1f0644560f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:10.818543 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35e67d-e499-46a0-a3f3-d1f0644560f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:10.819385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a27d7545-a01f-4c70-a063-09a434080ede None None] MFA Rules not processed for user `f54d72eea3684542879eab8ddd86aa90`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:10.821055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb35e67d-e499-46a0-a3f3-d1f0644560f1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:10.821168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a27d7545-a01f-4c70-a063-09a434080ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:10.865442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 973/1949] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:10 2022] POST /identity/v3/projects => generated 409 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:10.866807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35e67d-e499-46a0-a3f3-d1f0644560f1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.869025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bb35e67d-e499-46a0-a3f3-d1f0644560f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:10.924380 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 977/1950] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:10.925290 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d7545-a01f-4c70-a063-09a434080ede None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:10.951417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d7545-a01f-4c70-a063-09a434080ede None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.968082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb35e67d-e499-46a0-a3f3-d1f0644560f1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:10.973693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:10.975044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147665c-e2ba-48f8-a24a-d3994aebc783 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:10.976072 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147665c-e2ba-48f8-a24a-d3994aebc783 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:10.976434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147665c-e2ba-48f8-a24a-d3994aebc783 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:10.976835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147665c-e2ba-48f8-a24a-d3994aebc783 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.046920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb71587-e8be-4ca1-806d-1beed3fab8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.047681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb71587-e8be-4ca1-806d-1beed3fab8ee admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.048272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb71587-e8be-4ca1-806d-1beed3fab8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.048602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb71587-e8be-4ca1-806d-1beed3fab8ee admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.049979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb71587-e8be-4ca1-806d-1beed3fab8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.052286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb71587-e8be-4ca1-806d-1beed3fab8ee admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.067676 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147665c-e2ba-48f8-a24a-d3994aebc783 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.070681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147665c-e2ba-48f8-a24a-d3994aebc783 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.092886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 978/1951] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:10 2022] POST /identity/v3/users => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:11.093912 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb71587-e8be-4ca1-806d-1beed3fab8ee admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.096727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5fb71587-e8be-4ca1-806d-1beed3fab8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:11.107917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 974/1952] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:11.109492 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147665c-e2ba-48f8-a24a-d3994aebc783 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.127725 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb71587-e8be-4ca1-806d-1beed3fab8ee admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.133411 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c51c6-4758-4812-84a6-83d4405cc56d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.134594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c51c6-4758-4812-84a6-83d4405cc56d service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.135180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c51c6-4758-4812-84a6-83d4405cc56d service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.135485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c51c6-4758-4812-84a6-83d4405cc56d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.138665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147665c-e2ba-48f8-a24a-d3994aebc783 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.145486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40816d52-d1d3-4264-b3fd-5763513f883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.147183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40816d52-d1d3-4264-b3fd-5763513f883c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.147893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40816d52-d1d3-4264-b3fd-5763513f883c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.148418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40816d52-d1d3-4264-b3fd-5763513f883c admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.149477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40816d52-d1d3-4264-b3fd-5763513f883c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.151310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40816d52-d1d3-4264-b3fd-5763513f883c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.168979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 975/1953] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:11.171186 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40816d52-d1d3-4264-b3fd-5763513f883c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.185979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40816d52-d1d3-4264-b3fd-5763513f883c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.194458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca46f0-a0c1-4265-b27e-595b5260f252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.195409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca46f0-a0c1-4265-b27e-595b5260f252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.195698 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca46f0-a0c1-4265-b27e-595b5260f252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.196122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca46f0-a0c1-4265-b27e-595b5260f252 admin admin] PATH_INFO: `/v3/projects/f850eec9d0784fd78f0a59040954c880/users/fb6cf4300eca4927bd22fa84734c5284/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.222753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c51c6-4758-4812-84a6-83d4405cc56d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.231933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c51c6-4758-4812-84a6-83d4405cc56d service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.233990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca46f0-a0c1-4265-b27e-595b5260f252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f850eec9d0784fd78f0a59040954c880, user_id=fb6cf4300eca4927bd22fa84734c5284, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.240393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca46f0-a0c1-4265-b27e-595b5260f252 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.256519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 979/1954] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:11.278006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 976/1955] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:11 2022] PUT /identity/v3/projects/f850eec9d0784fd78f0a59040954c880/users/fb6cf4300eca4927bd22fa84734c5284/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:11.283355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c51c6-4758-4812-84a6-83d4405cc56d service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.295439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c51c6-4758-4812-84a6-83d4405cc56d service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.441238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b47897-e38b-4cc3-bbb2-beb01658900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.445207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b47897-e38b-4cc3-bbb2-beb01658900e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.445555 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b47897-e38b-4cc3-bbb2-beb01658900e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.445849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b47897-e38b-4cc3-bbb2-beb01658900e admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.448401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b47897-e38b-4cc3-bbb2-beb01658900e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.450296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b47897-e38b-4cc3-bbb2-beb01658900e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.477862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 980/1956] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:11.482628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca46f0-a0c1-4265-b27e-595b5260f252 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.505621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca46f0-a0c1-4265-b27e-595b5260f252 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.527280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a0a54-55ea-4c57-88ed-34e49a624887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.528599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a0a54-55ea-4c57-88ed-34e49a624887 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.528995 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a0a54-55ea-4c57-88ed-34e49a624887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.529401 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a0a54-55ea-4c57-88ed-34e49a624887 admin admin] PATH_INFO: `/v3/projects/f850eec9d0784fd78f0a59040954c880/users/fb6cf4300eca4927bd22fa84734c5284/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.534543 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a0a54-55ea-4c57-88ed-34e49a624887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f850eec9d0784fd78f0a59040954c880, user_id=fb6cf4300eca4927bd22fa84734c5284, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.541529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b47897-e38b-4cc3-bbb2-beb01658900e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.550623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a0a54-55ea-4c57-88ed-34e49a624887 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.570003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 977/1957] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:11 2022] PUT /identity/v3/projects/f850eec9d0784fd78f0a59040954c880/users/fb6cf4300eca4927bd22fa84734c5284/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:11.586121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081fef4-5c72-4281-bb5b-7481440916a9 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.586121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081fef4-5c72-4281-bb5b-7481440916a9 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.586290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081fef4-5c72-4281-bb5b-7481440916a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.615249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:11.639638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6081fef4-5c72-4281-bb5b-7481440916a9 None None] MFA Rules not processed for user `fb6cf4300eca4927bd22fa84734c5284`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:11.642606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6081fef4-5c72-4281-bb5b-7481440916a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:11.646037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b47897-e38b-4cc3-bbb2-beb01658900e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.664207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:11.721720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2960031-ae15-4727-b067-d2bf696b018e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.722849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2960031-ae15-4727-b067-d2bf696b018e service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.723242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2960031-ae15-4727-b067-d2bf696b018e service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.723611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2960031-ae15-4727-b067-d2bf696b018e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.733779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 978/1958] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:11 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:11.794080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081fef4-5c72-4281-bb5b-7481440916a9 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:11.800268 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2960031-ae15-4727-b067-d2bf696b018e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.801928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2960031-ae15-4727-b067-d2bf696b018e service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.804642 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081fef4-5c72-4281-bb5b-7481440916a9 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:11.808595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc113b0-b2b3-4fc0-adc9-d3cccb1fd94d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:11.809317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc113b0-b2b3-4fc0-adc9-d3cccb1fd94d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:11.809828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc113b0-b2b3-4fc0-adc9-d3cccb1fd94d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:11.810177 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc113b0-b2b3-4fc0-adc9-d3cccb1fd94d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:11.812111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 981/1959] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:28:11.821274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc113b0-b2b3-4fc0-adc9-d3cccb1fd94d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:11.822692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc113b0-b2b3-4fc0-adc9-d3cccb1fd94d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:11.833424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 979/1960] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:11 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:28:12.023157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2960031-ae15-4727-b067-d2bf696b018e service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:12.035131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2960031-ae15-4727-b067-d2bf696b018e service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:12.039212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20d2ed0-c9ec-4a0c-8a83-13716eed0fe8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:12.039895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d2ed0-c9ec-4a0c-8a83-13716eed0fe8 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:12.040178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d2ed0-c9ec-4a0c-8a83-13716eed0fe8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:12.040452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20d2ed0-c9ec-4a0c-8a83-13716eed0fe8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:12.122698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d2ed0-c9ec-4a0c-8a83-13716eed0fe8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:12.124566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20d2ed0-c9ec-4a0c-8a83-13716eed0fe8 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:12.138837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 982/1961] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:12 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:19.357105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc113b0-b2b3-4fc0-adc9-d3cccb1fd94d service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:19.434778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc113b0-b2b3-4fc0-adc9-d3cccb1fd94d service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:19.440425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:19.441306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.441619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.442095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] PATH_INFO: `/v3/users/2d0b3d8a81c34267ac6d4e2a68764c73` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.443307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d0b3d8a81c34267ac6d4e2a68764c73)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:19.445329 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:19.493514 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d0b3d8a81c34267ac6d4e2a68764c73'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:19.502049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d0b3d8a81c34267ac6d4e2a68764c73'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:19.502836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d0b3d8a81c34267ac6d4e2a68764c73'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:19.516307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d0b3d8a81c34267ac6d4e2a68764c73'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:19.523827 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d0b3d8a81c34267ac6d4e2a68764c73'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:19.530061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 980/1962] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:19 2022] DELETE /identity/v3/users/2d0b3d8a81c34267ac6d4e2a68764c73 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:19.535407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c790371-9b3c-49b4-868b-fa4d0824fc88 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.535992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c790371-9b3c-49b4-868b-fa4d0824fc88 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.536178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c790371-9b3c-49b4-868b-fa4d0824fc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.554135 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4c790371-9b3c-49b4-868b-fa4d0824fc88 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:19.555393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4c790371-9b3c-49b4-868b-fa4d0824fc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:19.606907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:19.627823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 983/1963] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:19.637328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:19.638799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:19.662747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:19.700282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2221d3-10bc-4cda-95ec-ee42046c9c2d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:19.704208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d4244-9dff-4014-803b-673b4c5b5f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:19.704906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d4244-9dff-4014-803b-673b4c5b5f1a service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:19.705194 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d4244-9dff-4014-803b-673b4c5b5f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:19.705502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94d4244-9dff-4014-803b-673b4c5b5f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:19.714497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d4244-9dff-4014-803b-673b4c5b5f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:19.715924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94d4244-9dff-4014-803b-673b4c5b5f1a service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:19.725987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 981/1964] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:20.026681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c790371-9b3c-49b4-868b-fa4d0824fc88 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.028361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4c790371-9b3c-49b4-868b-fa4d0824fc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:20.059534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c790371-9b3c-49b4-868b-fa4d0824fc88 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.065483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.066354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.066639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.066913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] PATH_INFO: `/v3/projects/2da864dadf614f15a06a00fded3fd819` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.068158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da864dadf614f15a06a00fded3fd819)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.070148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.122089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da864dadf614f15a06a00fded3fd819', 'request_id': 'req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.128705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da864dadf614f15a06a00fded3fd819', 'request_id': 'req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.136301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da864dadf614f15a06a00fded3fd819', 'request_id': 'req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.148336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] The token cache is being invalidate because project 2da864dadf614f15a06a00fded3fd819 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:20.148626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.149313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.151143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 984/1965] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:20 2022] DELETE /identity/v3/projects/2da864dadf614f15a06a00fded3fd819 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:20.154930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d4244-9dff-4014-803b-673b4c5b5f1a service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.165545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94d4244-9dff-4014-803b-673b4c5b5f1a service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.223710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.224605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.224894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.225175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] PATH_INFO: `/v3/users/d74694d70d0a4580b254ab6f598ac082` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.226420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d74694d70d0a4580b254ab6f598ac082)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.228573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.276435 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd74694d70d0a4580b254ab6f598ac082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.283302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd74694d70d0a4580b254ab6f598ac082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.283961 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd74694d70d0a4580b254ab6f598ac082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.296505 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd74694d70d0a4580b254ab6f598ac082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.305917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-eec3d06f-ca76-439b-8c0c-672ed7da082b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd74694d70d0a4580b254ab6f598ac082'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.312667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 982/1966] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:20 2022] DELETE /identity/v3/users/d74694d70d0a4580b254ab6f598ac082 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:20.604754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.607485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:20.635522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadaf67-19bd-41dd-8aea-a26e38e7aeec admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.703437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.704313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.704648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.704996 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] PATH_INFO: `/v3/projects/3ae607f184164acebfd8f382a9a64285` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.706288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae607f184164acebfd8f382a9a64285)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.708196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.757393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae607f184164acebfd8f382a9a64285', 'request_id': 'req-66ac8edc-d744-4e1c-8f50-066151187650'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.764204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae607f184164acebfd8f382a9a64285', 'request_id': 'req-66ac8edc-d744-4e1c-8f50-066151187650'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.771763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae607f184164acebfd8f382a9a64285', 'request_id': 'req-66ac8edc-d744-4e1c-8f50-066151187650'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.778877 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.783402 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] The token cache is being invalidate because project 3ae607f184164acebfd8f382a9a64285 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:20.783711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.784545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-66ac8edc-d744-4e1c-8f50-066151187650 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:20.786250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 985/1967] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:20 2022] DELETE /identity/v3/projects/3ae607f184164acebfd8f382a9a64285 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:20.791396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afdccc-0a9d-40cc-bbcc-06eba99e94a0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.791875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afdccc-0a9d-40cc-bbcc-06eba99e94a0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.791981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afdccc-0a9d-40cc-bbcc-06eba99e94a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.810301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-52afdccc-0a9d-40cc-bbcc-06eba99e94a0 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:20.811531 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-52afdccc-0a9d-40cc-bbcc-06eba99e94a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:20.853020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:20.876140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 983/1968] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:20.885427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7864481-03db-4069-9d30-f3b5a71e6bc1 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.885658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7864481-03db-4069-9d30-f3b5a71e6bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.885937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7864481-03db-4069-9d30-f3b5a71e6bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.904422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a7864481-03db-4069-9d30-f3b5a71e6bc1 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:20.905678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a7864481-03db-4069-9d30-f3b5a71e6bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:20.913765 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 986/1969] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:20.917976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afdccc-0a9d-40cc-bbcc-06eba99e94a0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.928778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afdccc-0a9d-40cc-bbcc-06eba99e94a0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.933406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ed0a7-f216-42eb-bab8-c9d9a58fe769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.934100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ed0a7-f216-42eb-bab8-c9d9a58fe769 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.934390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ed0a7-f216-42eb-bab8-c9d9a58fe769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.934674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ed0a7-f216-42eb-bab8-c9d9a58fe769 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.935413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ed0a7-f216-42eb-bab8-c9d9a58fe769 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:20.935734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ed0a7-f216-42eb-bab8-c9d9a58fe769 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.937605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899ed0a7-f216-42eb-bab8-c9d9a58fe769 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:20.950234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 984/1970] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:20.955333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7864481-03db-4069-9d30-f3b5a71e6bc1 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:20.964507 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7864481-03db-4069-9d30-f3b5a71e6bc1 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:20.974379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172db9b-1480-45f0-9557-0fce07805744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:20.975488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172db9b-1480-45f0-9557-0fce07805744 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:20.975968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172db9b-1480-45f0-9557-0fce07805744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:20.976438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a172db9b-1480-45f0-9557-0fce07805744 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:20.977743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172db9b-1480-45f0-9557-0fce07805744 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:20.981185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a172db9b-1480-45f0-9557-0fce07805744 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.011245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 987/1971] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:20 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:21.015554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ed0a7-f216-42eb-bab8-c9d9a58fe769 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.026948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ed0a7-f216-42eb-bab8-c9d9a58fe769 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.092937 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b248-c3ba-45cf-b55e-0e17d20bf73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.093763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b248-c3ba-45cf-b55e-0e17d20bf73c admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.094120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b248-c3ba-45cf-b55e-0e17d20bf73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.094474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db3b248-c3ba-45cf-b55e-0e17d20bf73c admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.096584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3b248-c3ba-45cf-b55e-0e17d20bf73c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.098970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db3b248-c3ba-45cf-b55e-0e17d20bf73c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.126011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 985/1972] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:21 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:21.127966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172db9b-1480-45f0-9557-0fce07805744 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.139139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a172db9b-1480-45f0-9557-0fce07805744 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.147567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048578be-f1ee-4679-bcc1-83cfb9e95730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.148764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048578be-f1ee-4679-bcc1-83cfb9e95730 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.149359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048578be-f1ee-4679-bcc1-83cfb9e95730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.150042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048578be-f1ee-4679-bcc1-83cfb9e95730 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.152005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048578be-f1ee-4679-bcc1-83cfb9e95730 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.154749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048578be-f1ee-4679-bcc1-83cfb9e95730 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.177696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 988/1973] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:21.181833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b248-c3ba-45cf-b55e-0e17d20bf73c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.194014 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db3b248-c3ba-45cf-b55e-0e17d20bf73c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.201233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b60c7a-654b-4297-85f7-3382bb6bb2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.201976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b60c7a-654b-4297-85f7-3382bb6bb2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.202260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b60c7a-654b-4297-85f7-3382bb6bb2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.202546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b60c7a-654b-4297-85f7-3382bb6bb2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.203285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b60c7a-654b-4297-85f7-3382bb6bb2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.205651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b60c7a-654b-4297-85f7-3382bb6bb2a5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.216491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 986/1974] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:21.221479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048578be-f1ee-4679-bcc1-83cfb9e95730 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.239092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048578be-f1ee-4679-bcc1-83cfb9e95730 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.248347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590af1be-c4cc-4094-afb1-5609a43e41da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.252051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590af1be-c4cc-4094-afb1-5609a43e41da admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.252516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590af1be-c4cc-4094-afb1-5609a43e41da admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.252853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590af1be-c4cc-4094-afb1-5609a43e41da admin admin] PATH_INFO: `/v3/projects/9fa24df0f2304145aafafbd4c641111f/users/f1af03c329cf4738ab29ddd81851834a/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.271584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590af1be-c4cc-4094-afb1-5609a43e41da admin admin] RBAC: Authorizing `identity:create_grant(project_id=9fa24df0f2304145aafafbd4c641111f, user_id=f1af03c329cf4738ab29ddd81851834a, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.274269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590af1be-c4cc-4094-afb1-5609a43e41da admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.288002 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 989/1975] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:21 2022] PUT /identity/v3/projects/9fa24df0f2304145aafafbd4c641111f/users/f1af03c329cf4738ab29ddd81851834a/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:21.298665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624fbe0-18b8-4624-bba3-7c7045642a75 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.298957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624fbe0-18b8-4624-bba3-7c7045642a75 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.299235 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624fbe0-18b8-4624-bba3-7c7045642a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.328592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2624fbe0-18b8-4624-bba3-7c7045642a75 None None] MFA Rules not processed for user `f1af03c329cf4738ab29ddd81851834a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:21.330600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2624fbe0-18b8-4624-bba3-7c7045642a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:21.404000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 987/1976] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:21.408607 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590af1be-c4cc-4094-afb1-5609a43e41da admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.472122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590af1be-c4cc-4094-afb1-5609a43e41da admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.477284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521ff9e-4380-4646-af3c-b072e0388eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.478017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521ff9e-4380-4646-af3c-b072e0388eb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.478302 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521ff9e-4380-4646-af3c-b072e0388eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.478578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521ff9e-4380-4646-af3c-b072e0388eb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.479283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521ff9e-4380-4646-af3c-b072e0388eb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.481218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8521ff9e-4380-4646-af3c-b072e0388eb7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.520314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 990/1977] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:21 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:21.525032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624fbe0-18b8-4624-bba3-7c7045642a75 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.536969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2624fbe0-18b8-4624-bba3-7c7045642a75 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.582043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.614328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d6b7f-8934-41ec-98f5-be20b93bc1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.615042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d6b7f-8934-41ec-98f5-be20b93bc1fd admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.615366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d6b7f-8934-41ec-98f5-be20b93bc1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.615637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d6b7f-8934-41ec-98f5-be20b93bc1fd admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.616347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d6b7f-8934-41ec-98f5-be20b93bc1fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.618332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132d6b7f-8934-41ec-98f5-be20b93bc1fd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.639122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 988/1978] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:21 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:21.643829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521ff9e-4380-4646-af3c-b072e0388eb7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.655862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521ff9e-4380-4646-af3c-b072e0388eb7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.660870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acc92b-d791-4051-8682-41beed6cac35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.661657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acc92b-d791-4051-8682-41beed6cac35 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.662000 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acc92b-d791-4051-8682-41beed6cac35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.662355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acc92b-d791-4051-8682-41beed6cac35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.664707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acc92b-d791-4051-8682-41beed6cac35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.667044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acc92b-d791-4051-8682-41beed6cac35 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.680029 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 991/1979] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:21.683893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d6b7f-8934-41ec-98f5-be20b93bc1fd admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.695947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d6b7f-8934-41ec-98f5-be20b93bc1fd admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.701129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5750ee-13a0-45fe-baac-515f5867cd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.702120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5750ee-13a0-45fe-baac-515f5867cd14 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.702335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5750ee-13a0-45fe-baac-515f5867cd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.702707 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5750ee-13a0-45fe-baac-515f5867cd14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.706050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5750ee-13a0-45fe-baac-515f5867cd14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.706050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5750ee-13a0-45fe-baac-515f5867cd14 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.720295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 989/1980] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:21.721772 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acc92b-d791-4051-8682-41beed6cac35 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.732814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acc92b-d791-4051-8682-41beed6cac35 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.738372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa647d9e-85c4-4bf3-aa36-754b104a444f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.739274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa647d9e-85c4-4bf3-aa36-754b104a444f admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.739566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa647d9e-85c4-4bf3-aa36-754b104a444f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.739836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa647d9e-85c4-4bf3-aa36-754b104a444f admin admin] PATH_INFO: `/v3/projects/eebdcb47c1c24269bb72643cbaf06410/users/33d1d99271654ef08acc835b15e46084/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.756972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa647d9e-85c4-4bf3-aa36-754b104a444f admin admin] RBAC: Authorizing `identity:create_grant(project_id=eebdcb47c1c24269bb72643cbaf06410, user_id=33d1d99271654ef08acc835b15e46084, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.759734 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa647d9e-85c4-4bf3-aa36-754b104a444f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:21.775314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 992/1981] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:21 2022] PUT /identity/v3/projects/eebdcb47c1c24269bb72643cbaf06410/users/33d1d99271654ef08acc835b15e46084/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:21.786764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac661ea-da84-417e-ba1f-eefc640a9486 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.787066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac661ea-da84-417e-ba1f-eefc640a9486 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.787341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac661ea-da84-417e-ba1f-eefc640a9486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.819342 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-dac661ea-da84-417e-ba1f-eefc640a9486 None None] MFA Rules not processed for user `33d1d99271654ef08acc835b15e46084`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:21.820952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-dac661ea-da84-417e-ba1f-eefc640a9486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:21.891284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 990/1982] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:21.896217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa647d9e-85c4-4bf3-aa36-754b104a444f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:21.952180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:21.968793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa647d9e-85c4-4bf3-aa36-754b104a444f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:21.974095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a37da-86dd-4e8b-bda4-fc3e686c5647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:21.974763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a37da-86dd-4e8b-bda4-fc3e686c5647 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:21.975084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a37da-86dd-4e8b-bda4-fc3e686c5647 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:21.975363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a37da-86dd-4e8b-bda4-fc3e686c5647 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:21.976070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a37da-86dd-4e8b-bda4-fc3e686c5647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:21.978127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a37da-86dd-4e8b-bda4-fc3e686c5647 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.004345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 993/1983] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:21 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:22.008471 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac661ea-da84-417e-ba1f-eefc640a9486 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.020858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac661ea-da84-417e-ba1f-eefc640a9486 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.082077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce696ac6-21ed-464e-90be-ebf2977b693b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.082751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce696ac6-21ed-464e-90be-ebf2977b693b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.083038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce696ac6-21ed-464e-90be-ebf2977b693b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.083312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce696ac6-21ed-464e-90be-ebf2977b693b admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.084001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce696ac6-21ed-464e-90be-ebf2977b693b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.085953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce696ac6-21ed-464e-90be-ebf2977b693b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.106469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 991/1984] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:22 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:22.110994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a37da-86dd-4e8b-bda4-fc3e686c5647 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.123437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a37da-86dd-4e8b-bda4-fc3e686c5647 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.132252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53380b-7767-4995-9852-26c599aac819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.132956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53380b-7767-4995-9852-26c599aac819 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.133240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53380b-7767-4995-9852-26c599aac819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.133556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf53380b-7767-4995-9852-26c599aac819 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.134294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53380b-7767-4995-9852-26c599aac819 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.135910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf53380b-7767-4995-9852-26c599aac819 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.149175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 994/1985] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:22.153583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce696ac6-21ed-464e-90be-ebf2977b693b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.164124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce696ac6-21ed-464e-90be-ebf2977b693b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.168744 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a68a7-05d4-4630-877f-45cd7e21eee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.169811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a68a7-05d4-4630-877f-45cd7e21eee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.170099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a68a7-05d4-4630-877f-45cd7e21eee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.170377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23a68a7-05d4-4630-877f-45cd7e21eee2 admin admin] PATH_INFO: `/v3/projects/ff1afcf633e24697bdf5e630092d24c3/users/4d5c2a75614b47e6b54c58e2a9cf5d30/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.187080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a68a7-05d4-4630-877f-45cd7e21eee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1afcf633e24697bdf5e630092d24c3, user_id=4d5c2a75614b47e6b54c58e2a9cf5d30, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.191573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23a68a7-05d4-4630-877f-45cd7e21eee2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.207529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 992/1986] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:22 2022] PUT /identity/v3/projects/ff1afcf633e24697bdf5e630092d24c3/users/4d5c2a75614b47e6b54c58e2a9cf5d30/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:22.211364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53380b-7767-4995-9852-26c599aac819 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.222459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf53380b-7767-4995-9852-26c599aac819 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.316309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836a8a6-e97a-4db2-96ee-37c1131145c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.317007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836a8a6-e97a-4db2-96ee-37c1131145c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.317298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836a8a6-e97a-4db2-96ee-37c1131145c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.317613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836a8a6-e97a-4db2-96ee-37c1131145c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.318363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836a8a6-e97a-4db2-96ee-37c1131145c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.320124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836a8a6-e97a-4db2-96ee-37c1131145c5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.334051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 995/1987] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:22.337358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a68a7-05d4-4630-877f-45cd7e21eee2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.355848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23a68a7-05d4-4630-877f-45cd7e21eee2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.362807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff7f12-8efc-4ec5-950f-bef4a397e5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.363776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff7f12-8efc-4ec5-950f-bef4a397e5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.364183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff7f12-8efc-4ec5-950f-bef4a397e5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.364391 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bff7f12-8efc-4ec5-950f-bef4a397e5a9 admin admin] PATH_INFO: `/v3/projects/ff1afcf633e24697bdf5e630092d24c3/users/4d5c2a75614b47e6b54c58e2a9cf5d30/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.366491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff7f12-8efc-4ec5-950f-bef4a397e5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff1afcf633e24697bdf5e630092d24c3, user_id=4d5c2a75614b47e6b54c58e2a9cf5d30, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.373190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bff7f12-8efc-4ec5-950f-bef4a397e5a9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.379936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:22.392195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 993/1988] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:22 2022] PUT /identity/v3/projects/ff1afcf633e24697bdf5e630092d24c3/users/4d5c2a75614b47e6b54c58e2a9cf5d30/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:22.398438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092eab51-4e0b-443e-b204-e8b430bb703b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.398846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092eab51-4e0b-443e-b204-e8b430bb703b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.399690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092eab51-4e0b-443e-b204-e8b430bb703b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.429359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-092eab51-4e0b-443e-b204-e8b430bb703b None None] MFA Rules not processed for user `4d5c2a75614b47e6b54c58e2a9cf5d30`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:22.430755 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-092eab51-4e0b-443e-b204-e8b430bb703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:22.506752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 996/1989] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:22 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:22.519599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff7f12-8efc-4ec5-950f-bef4a397e5a9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:22.521345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6bff7f12-8efc-4ec5-950f-bef4a397e5a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:22.581662 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bff7f12-8efc-4ec5-950f-bef4a397e5a9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:22.585905 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2101428b-952e-4283-88a4-4d84c484cfa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:22.586560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101428b-952e-4283-88a4-4d84c484cfa9 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:22.586844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101428b-952e-4283-88a4-4d84c484cfa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:22.587121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2101428b-952e-4283-88a4-4d84c484cfa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:22.643734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101428b-952e-4283-88a4-4d84c484cfa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:22.645179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2101428b-952e-4283-88a4-4d84c484cfa9 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:22.654096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 994/1990] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:25.240933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092eab51-4e0b-443e-b204-e8b430bb703b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.242468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-092eab51-4e0b-443e-b204-e8b430bb703b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:25.256884 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092eab51-4e0b-443e-b204-e8b430bb703b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.262355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e98ac9-8f61-4397-801e-43c796db978f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.263076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98ac9-8f61-4397-801e-43c796db978f service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.263548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98ac9-8f61-4397-801e-43c796db978f service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.263852 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e98ac9-8f61-4397-801e-43c796db978f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.320175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:25.360403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e98ac9-8f61-4397-801e-43c796db978f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:25.368364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e98ac9-8f61-4397-801e-43c796db978f service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:25.380606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 997/1991] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:25.640579 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9cdc9-cf56-4cd4-828a-3cc32af71282 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.641856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9cdc9-cf56-4cd4-828a-3cc32af71282 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.642333 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd9cdc9-cf56-4cd4-828a-3cc32af71282 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.644984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 995/1992] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:28:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:28:25.708247 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd0bbb-1019-4c53-b13e-94ad4aa1dc57 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.708794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd0bbb-1019-4c53-b13e-94ad4aa1dc57 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.709115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd0bbb-1019-4c53-b13e-94ad4aa1dc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.758740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8cbd0bbb-1019-4c53-b13e-94ad4aa1dc57 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:25.759981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8cbd0bbb-1019-4c53-b13e-94ad4aa1dc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:25.786820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9cdc9-cf56-4cd4-828a-3cc32af71282 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:25.788452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1bd9cdc9-cf56-4cd4-828a-3cc32af71282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:25.928218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 998/1993] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:28:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:25.930412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd9cdc9-cf56-4cd4-828a-3cc32af71282 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:25.951636 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:25.952665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:25.953817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:25.954340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] PATH_INFO: `/v3/users/33bf221cf90b48b8ac5297bddba1bbb2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:25.957156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33bf221cf90b48b8ac5297bddba1bbb2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:25.961936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:26.098286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33bf221cf90b48b8ac5297bddba1bbb2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.116139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:26.120029 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33bf221cf90b48b8ac5297bddba1bbb2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.120983 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33bf221cf90b48b8ac5297bddba1bbb2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.169080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33bf221cf90b48b8ac5297bddba1bbb2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.182358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33bf221cf90b48b8ac5297bddba1bbb2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.215340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 996/1994] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:25 2022] DELETE /identity/v3/users/33bf221cf90b48b8ac5297bddba1bbb2 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:26.219405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8b37-9277-491b-b51d-4ff62a002573 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:26.219711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8b37-9277-491b-b51d-4ff62a002573 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:26.219999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8b37-9277-491b-b51d-4ff62a002573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:26.256001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e03b8b37-9277-491b-b51d-4ff62a002573 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:26.257283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e03b8b37-9277-491b-b51d-4ff62a002573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:26.396494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 999/1995] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:26.410341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:26.411974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:26.414194 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8b37-9277-491b-b51d-4ff62a002573 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:26.415801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e03b8b37-9277-491b-b51d-4ff62a002573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:26.495821 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8698d652-2ae5-4a54-94de-a557d10c41b4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:26.499203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8b37-9277-491b-b51d-4ff62a002573 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:26.501394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42bb05-3522-4522-bf8f-7fb19f77e41d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:26.502242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42bb05-3522-4522-bf8f-7fb19f77e41d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:26.503493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42bb05-3522-4522-bf8f-7fb19f77e41d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:26.504188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce42bb05-3522-4522-bf8f-7fb19f77e41d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:26.505138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:26.506133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:26.506831 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:26.507381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:26.510021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:26.527079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42bb05-3522-4522-bf8f-7fb19f77e41d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:26.528750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce42bb05-3522-4522-bf8f-7fb19f77e41d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:26.543138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 997/1996] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:26.548230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:26.555980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42bb05-3522-4522-bf8f-7fb19f77e41d service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:26.557464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ce42bb05-3522-4522-bf8f-7fb19f77e41d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:26.595663 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:26.597168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:26.610316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1000/1997] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:28:26.651643 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce42bb05-3522-4522-bf8f-7fb19f77e41d service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:26.658822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac14a69-6da7-4794-a7a2-9882dc4feff2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:26.659509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac14a69-6da7-4794-a7a2-9882dc4feff2 service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:26.659794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac14a69-6da7-4794-a7a2-9882dc4feff2 service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:26.660105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac14a69-6da7-4794-a7a2-9882dc4feff2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:26.662401 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9ac14a69-6da7-4794-a7a2-9882dc4feff2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:26.678181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac14a69-6da7-4794-a7a2-9882dc4feff2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:26.679919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac14a69-6da7-4794-a7a2-9882dc4feff2 service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:26.695817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 998/1998] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:26.848258 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:26.849829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:26.864211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aba43b9-d036-4dc3-9005-df34372a842a service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:26.869422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:26.870265 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:26.870551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:26.870829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] PATH_INFO: `/v3/projects/1a5fb6af1b20439589ca660c05ba2b62` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:26.872073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a5fb6af1b20439589ca660c05ba2b62)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:26.873981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:26.927013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a5fb6af1b20439589ca660c05ba2b62', 'request_id': 'req-7ac1bd9d-2075-4430-8f58-df36518528c7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.933476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a5fb6af1b20439589ca660c05ba2b62', 'request_id': 'req-7ac1bd9d-2075-4430-8f58-df36518528c7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.941982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a5fb6af1b20439589ca660c05ba2b62', 'request_id': 'req-7ac1bd9d-2075-4430-8f58-df36518528c7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.953315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] The token cache is being invalidate because project 1a5fb6af1b20439589ca660c05ba2b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:26.953768 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.954297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:26.956371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1001/1999] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:26 2022] DELETE /identity/v3/projects/1a5fb6af1b20439589ca660c05ba2b62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:26.961102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac14a69-6da7-4794-a7a2-9882dc4feff2 service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:26.973036 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac14a69-6da7-4794-a7a2-9882dc4feff2 service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.032502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.034529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.035041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.035380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] PATH_INFO: `/v3/users/f54d72eea3684542879eab8ddd86aa90` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.036867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=f54d72eea3684542879eab8ddd86aa90)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:27.039449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:27.041559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:27.094037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f54d72eea3684542879eab8ddd86aa90'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.101620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f54d72eea3684542879eab8ddd86aa90'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.102308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f54d72eea3684542879eab8ddd86aa90'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.114649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f54d72eea3684542879eab8ddd86aa90'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.121796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f54d72eea3684542879eab8ddd86aa90'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.128137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 999/2000] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:26 2022] DELETE /identity/v3/users/f54d72eea3684542879eab8ddd86aa90 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:27.379067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:27.380787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:27.407881 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1bd9d-2075-4430-8f58-df36518528c7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.495368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.496346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.496721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.497082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] PATH_INFO: `/v3/users/38845d328a544efbae036e0f726464be` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.498335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38845d328a544efbae036e0f726464be)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:27.500585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:27.510240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:27.511682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:27.542847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41d03d9-16e2-458d-8e58-095e9a5bf7cd admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.555804 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.556891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.557216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.557639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] PATH_INFO: `/v3/projects/c72c03b6e3d84feab93c73129e4f720a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.559070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c72c03b6e3d84feab93c73129e4f720a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:27.567165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:27.596863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38845d328a544efbae036e0f726464be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.598951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:27.615889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38845d328a544efbae036e0f726464be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.618198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38845d328a544efbae036e0f726464be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.633713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38845d328a544efbae036e0f726464be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.657667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38845d328a544efbae036e0f726464be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.662313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c72c03b6e3d84feab93c73129e4f720a', 'request_id': 'req-3ebb2a8c-a552-4128-b99a-2c79f85d311b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.669552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c72c03b6e3d84feab93c73129e4f720a', 'request_id': 'req-3ebb2a8c-a552-4128-b99a-2c79f85d311b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.674652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1002/2001] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:27 2022] DELETE /identity/v3/users/38845d328a544efbae036e0f726464be => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:27.675655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:27.677229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:27.682148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c72c03b6e3d84feab93c73129e4f720a', 'request_id': 'req-3ebb2a8c-a552-4128-b99a-2c79f85d311b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.696315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] The token cache is being invalidate because project c72c03b6e3d84feab93c73129e4f720a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:27.696622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.697614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3ebb2a8c-a552-4128-b99a-2c79f85d311b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.700394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1000/2002] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:27 2022] DELETE /identity/v3/projects/c72c03b6e3d84feab93c73129e4f720a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:27.701848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989bb60-0412-4d93-800a-11b6a7e2b0a1 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.702149 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989bb60-0412-4d93-800a-11b6a7e2b0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.702436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d989bb60-0412-4d93-800a-11b6a7e2b0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.724737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d989bb60-0412-4d93-800a-11b6a7e2b0a1 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:27.726024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d989bb60-0412-4d93-800a-11b6a7e2b0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:27.753183 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6820787e-a298-4aee-b705-0402a7f56301 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.759416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:27.759416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.759642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.760061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] PATH_INFO: `/v3/users/34ee97f7c30d44749720076b02882892` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.761044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=34ee97f7c30d44749720076b02882892)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:27.763056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:27.820477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1001/2003] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:27.823720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2aa63-b004-459b-af30-dd59f0cd8086 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.823720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2aa63-b004-459b-af30-dd59f0cd8086 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.823720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c2aa63-b004-459b-af30-dd59f0cd8086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.826413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34ee97f7c30d44749720076b02882892'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.836474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:27.837438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34ee97f7c30d44749720076b02882892'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.838698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34ee97f7c30d44749720076b02882892'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.849886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-68c2aa63-b004-459b-af30-dd59f0cd8086 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:27.851343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-68c2aa63-b004-459b-af30-dd59f0cd8086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:27.858683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34ee97f7c30d44749720076b02882892'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.862590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1002/2004] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:27.866799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2aa63-b004-459b-af30-dd59f0cd8086 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:27.869695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-68c2aa63-b004-459b-af30-dd59f0cd8086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:27.877855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-877b4064-be2e-455d-93c7-0d6c83586c0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34ee97f7c30d44749720076b02882892'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:27.888254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1003/2005] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:27 2022] DELETE /identity/v3/users/34ee97f7c30d44749720076b02882892 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:27.888254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaef377-4c93-4dc2-8ea0-b2e27c635dba None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.888254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaef377-4c93-4dc2-8ea0-b2e27c635dba None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.888254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaef377-4c93-4dc2-8ea0-b2e27c635dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:27.908069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bfaef377-4c93-4dc2-8ea0-b2e27c635dba None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:27.911587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bfaef377-4c93-4dc2-8ea0-b2e27c635dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:27.939622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c2aa63-b004-459b-af30-dd59f0cd8086 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:27.961658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:27.991575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1004/2006] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:27.993239 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b72ac5-c142-496d-b038-de071eb28733 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:27.993710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b72ac5-c142-496d-b038-de071eb28733 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:27.993989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b72ac5-c142-496d-b038-de071eb28733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.002535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.003158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.003544 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.003830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.005740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:28.013254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a8b72ac5-c142-496d-b038-de071eb28733 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:28.014478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a8b72ac5-c142-496d-b038-de071eb28733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:28.024609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1005/2007] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:28.025228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b72ac5-c142-496d-b038-de071eb28733 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.027529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.029040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.037051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b72ac5-c142-496d-b038-de071eb28733 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.041165 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1003/2008] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:28.042153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.043463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:28.044381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c06d3-61d7-4cb6-b032-da5cfe12ad79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.045856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c06d3-61d7-4cb6-b032-da5cfe12ad79 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.046171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c06d3-61d7-4cb6-b032-da5cfe12ad79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.046447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2c06d3-61d7-4cb6-b032-da5cfe12ad79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.047288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c06d3-61d7-4cb6-b032-da5cfe12ad79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:28.047598 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c06d3-61d7-4cb6-b032-da5cfe12ad79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.050171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2c06d3-61d7-4cb6-b032-da5cfe12ad79 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.056568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c1df9e-3df4-4b35-b4ea-a712b3092786 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.070563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1006/2009] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:28.071519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c06d3-61d7-4cb6-b032-da5cfe12ad79 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.079103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad8e9d-0536-40ac-9d5a-50120bcf8371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.079816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad8e9d-0536-40ac-9d5a-50120bcf8371 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.080190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad8e9d-0536-40ac-9d5a-50120bcf8371 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.081272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad8e9d-0536-40ac-9d5a-50120bcf8371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.094975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2c06d3-61d7-4cb6-b032-da5cfe12ad79 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.101444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ac65f-0254-41fa-9494-4171d18cdb67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.103281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad8e9d-0536-40ac-9d5a-50120bcf8371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.104929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad8e9d-0536-40ac-9d5a-50120bcf8371 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.107583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ac65f-0254-41fa-9494-4171d18cdb67 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.107803 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ac65f-0254-41fa-9494-4171d18cdb67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.108228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ac65f-0254-41fa-9494-4171d18cdb67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.109171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ac65f-0254-41fa-9494-4171d18cdb67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.113347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ac65f-0254-41fa-9494-4171d18cdb67 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.126453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1004/2010] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:28.168434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1007/2011] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:28 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:28.172632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad8e9d-0536-40ac-9d5a-50120bcf8371 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.197649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad8e9d-0536-40ac-9d5a-50120bcf8371 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.273468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c458d-f987-4fbc-b916-0b936d7a9b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.274480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c458d-f987-4fbc-b916-0b936d7a9b67 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.274972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c458d-f987-4fbc-b916-0b936d7a9b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.275171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969c458d-f987-4fbc-b916-0b936d7a9b67 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.275988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c458d-f987-4fbc-b916-0b936d7a9b67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.278641 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969c458d-f987-4fbc-b916-0b936d7a9b67 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.303739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1005/2012] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:28 2022] POST /identity/v3/users => generated 293 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:28.307750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ac65f-0254-41fa-9494-4171d18cdb67 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.320296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ac65f-0254-41fa-9494-4171d18cdb67 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.331870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e21d02-2123-464d-845d-6b4427dda03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.332576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e21d02-2123-464d-845d-6b4427dda03f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.333212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e21d02-2123-464d-845d-6b4427dda03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.333212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e21d02-2123-464d-845d-6b4427dda03f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.334013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e21d02-2123-464d-845d-6b4427dda03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.335692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e21d02-2123-464d-845d-6b4427dda03f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.366933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1008/2013] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:28.369311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c458d-f987-4fbc-b916-0b936d7a9b67 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.385905 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969c458d-f987-4fbc-b916-0b936d7a9b67 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.395465 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1a52d-bb7b-4600-be9c-cbdd1f2843e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.396155 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1a52d-bb7b-4600-be9c-cbdd1f2843e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.396438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1a52d-bb7b-4600-be9c-cbdd1f2843e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.396713 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1a52d-bb7b-4600-be9c-cbdd1f2843e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.397626 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1a52d-bb7b-4600-be9c-cbdd1f2843e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.399259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1a52d-bb7b-4600-be9c-cbdd1f2843e2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.411525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1006/2014] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:28.416079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e21d02-2123-464d-845d-6b4427dda03f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.433560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e21d02-2123-464d-845d-6b4427dda03f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.439618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1e425-1a2f-4c4a-827e-772944b9048a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.440863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1e425-1a2f-4c4a-827e-772944b9048a admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.441157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1e425-1a2f-4c4a-827e-772944b9048a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.441468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1e425-1a2f-4c4a-827e-772944b9048a admin admin] PATH_INFO: `/v3/projects/c1814ed244354c17b1ac7b1089a74bb7/users/e010b6d917ca464dbba63d822cfcff1d/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.459332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1e425-1a2f-4c4a-827e-772944b9048a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1814ed244354c17b1ac7b1089a74bb7, user_id=e010b6d917ca464dbba63d822cfcff1d, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.461809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1e425-1a2f-4c4a-827e-772944b9048a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.491410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1009/2015] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:28 2022] PUT /identity/v3/projects/c1814ed244354c17b1ac7b1089a74bb7/users/e010b6d917ca464dbba63d822cfcff1d/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:28.493914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f21bd4-8f4f-43f9-8a1e-3f9bbac174cf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.494208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f21bd4-8f4f-43f9-8a1e-3f9bbac174cf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.494514 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f21bd4-8f4f-43f9-8a1e-3f9bbac174cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.526390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-48f21bd4-8f4f-43f9-8a1e-3f9bbac174cf None None] MFA Rules not processed for user `e010b6d917ca464dbba63d822cfcff1d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:28.527673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-48f21bd4-8f4f-43f9-8a1e-3f9bbac174cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:28.540173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:28.579560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1e425-1a2f-4c4a-827e-772944b9048a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.582014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6ef1e425-1a2f-4c4a-827e-772944b9048a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:28.648191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1007/2016] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:28.652329 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f21bd4-8f4f-43f9-8a1e-3f9bbac174cf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.662173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1e425-1a2f-4c4a-827e-772944b9048a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.668121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f21bd4-8f4f-43f9-8a1e-3f9bbac174cf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.669297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.670204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.670490 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.670768 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] PATH_INFO: `/v3/projects/0122b63b80e34f299ffd0afd403dea34` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.671967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0122b63b80e34f299ffd0afd403dea34)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.673929 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.674257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6c298-89e7-4ba2-ba9f-993e122ca91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.675574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6c298-89e7-4ba2-ba9f-993e122ca91d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.675574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6c298-89e7-4ba2-ba9f-993e122ca91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.675775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a6c298-89e7-4ba2-ba9f-993e122ca91d admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.676442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6c298-89e7-4ba2-ba9f-993e122ca91d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.678425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a6c298-89e7-4ba2-ba9f-993e122ca91d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.713244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1008/2017] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:28 2022] POST /identity/v3/projects => generated 381 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:28.714360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6c298-89e7-4ba2-ba9f-993e122ca91d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.734407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0122b63b80e34f299ffd0afd403dea34', 'request_id': 'req-084522da-5f50-4285-b267-926d76d3ee01'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:28.743354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0122b63b80e34f299ffd0afd403dea34', 'request_id': 'req-084522da-5f50-4285-b267-926d76d3ee01'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:28.753284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0122b63b80e34f299ffd0afd403dea34', 'request_id': 'req-084522da-5f50-4285-b267-926d76d3ee01'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:28.762143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:28.768908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] The token cache is being invalidate because project 0122b63b80e34f299ffd0afd403dea34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:28.769729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:28.770454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:28.772650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1010/2018] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:28 2022] DELETE /identity/v3/projects/0122b63b80e34f299ffd0afd403dea34 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:28.776714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.794307 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:28.808550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a6c298-89e7-4ba2-ba9f-993e122ca91d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.899572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91852a-91cd-4e82-9819-b85a8308e59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.900590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91852a-91cd-4e82-9819-b85a8308e59a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.901064 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91852a-91cd-4e82-9819-b85a8308e59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.901558 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91852a-91cd-4e82-9819-b85a8308e59a admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.902469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91852a-91cd-4e82-9819-b85a8308e59a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.904892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a91852a-91cd-4e82-9819-b85a8308e59a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.934221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1009/2019] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:28 2022] POST /identity/v3/users => generated 297 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:28.936826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91852a-91cd-4e82-9819-b85a8308e59a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:28.942452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8a91852a-91cd-4e82-9819-b85a8308e59a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:28.972424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084522da-5f50-4285-b267-926d76d3ee01 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.980419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.981172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91852a-91cd-4e82-9819-b85a8308e59a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:28.981330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.982038 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.982405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] PATH_INFO: `/v3/users/fb6cf4300eca4927bd22fa84734c5284` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:28.984145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb6cf4300eca4927bd22fa84734c5284)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:28.986296 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:28.994934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:28.996900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:28.997339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:28.998252 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] PATH_INFO: `/v3/projects/d0dbb756b60f4ad9a4dcf930a4c1c090` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.001971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0dbb756b60f4ad9a4dcf930a4c1c090)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.008414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.036980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:29.056606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb6cf4300eca4927bd22fa84734c5284'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.067585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb6cf4300eca4927bd22fa84734c5284'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.068867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb6cf4300eca4927bd22fa84734c5284'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.082866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb6cf4300eca4927bd22fa84734c5284'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.088908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0dbb756b60f4ad9a4dcf930a4c1c090', 'request_id': 'req-c9117088-b2ac-489c-b201-2d186d519cae'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.100053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0dbb756b60f4ad9a4dcf930a4c1c090', 'request_id': 'req-c9117088-b2ac-489c-b201-2d186d519cae'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.108332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb6cf4300eca4927bd22fa84734c5284'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.113943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0dbb756b60f4ad9a4dcf930a4c1c090', 'request_id': 'req-c9117088-b2ac-489c-b201-2d186d519cae'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.122430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1011/2020] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:28 2022] DELETE /identity/v3/users/fb6cf4300eca4927bd22fa84734c5284 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:29.122430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.122798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:29.144301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] The token cache is being invalidate because project d0dbb756b60f4ad9a4dcf930a4c1c090 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:29.144718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.146822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.149591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1010/2021] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:28 2022] DELETE /identity/v3/projects/d0dbb756b60f4ad9a4dcf930a4c1c090 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:29.154967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.157068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:29.279917 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b823c-3726-42a8-b85f-5c22cc03d63a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.285835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffcd4f-ae05-4fc5-a459-5a953bc874a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.286649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffcd4f-ae05-4fc5-a459-5a953bc874a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.287045 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffcd4f-ae05-4fc5-a459-5a953bc874a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.287322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ffcd4f-ae05-4fc5-a459-5a953bc874a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.288063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffcd4f-ae05-4fc5-a459-5a953bc874a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.290176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ffcd4f-ae05-4fc5-a459-5a953bc874a5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.305418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1012/2022] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:29.306662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffcd4f-ae05-4fc5-a459-5a953bc874a5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.310008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9117088-b2ac-489c-b201-2d186d519cae admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.312801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-76ffcd4f-ae05-4fc5-a459-5a953bc874a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:29.319788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.325709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.325709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.325709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] PATH_INFO: `/v3/users/4afe6a57c79a4c959ae81b0201540985` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.325709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=4afe6a57c79a4c959ae81b0201540985)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.327653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.334004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ffcd4f-ae05-4fc5-a459-5a953bc874a5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.343032 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b02d3-7d2f-45af-9020-c516746027d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.344925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b02d3-7d2f-45af-9020-c516746027d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.347493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b02d3-7d2f-45af-9020-c516746027d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.348064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b02d3-7d2f-45af-9020-c516746027d8 admin admin] PATH_INFO: `/v3/projects/441ffbe20a794264a493c820b420c250/users/5d40b53d8b37452797f1539b4acadbf4/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.370069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:29.375405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b02d3-7d2f-45af-9020-c516746027d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441ffbe20a794264a493c820b420c250, user_id=5d40b53d8b37452797f1539b4acadbf4, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.379954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b02d3-7d2f-45af-9020-c516746027d8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.414090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1013/2023] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:29 2022] PUT /identity/v3/projects/441ffbe20a794264a493c820b420c250/users/5d40b53d8b37452797f1539b4acadbf4/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:29.415685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4afe6a57c79a4c959ae81b0201540985'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.417881 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b02d3-7d2f-45af-9020-c516746027d8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.427180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4afe6a57c79a4c959ae81b0201540985'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.427783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4afe6a57c79a4c959ae81b0201540985'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.446792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4afe6a57c79a4c959ae81b0201540985'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.465021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4afe6a57c79a4c959ae81b0201540985'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.473937 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1011/2024] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:29 2022] DELETE /identity/v3/users/4afe6a57c79a4c959ae81b0201540985 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:29.504662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b02d3-7d2f-45af-9020-c516746027d8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.555553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.557426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:29.632473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694e8c4-7954-411a-87df-6f50cf8cef90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.634843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694e8c4-7954-411a-87df-6f50cf8cef90 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.635474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694e8c4-7954-411a-87df-6f50cf8cef90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.635923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d694e8c4-7954-411a-87df-6f50cf8cef90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.637534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694e8c4-7954-411a-87df-6f50cf8cef90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.640263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d694e8c4-7954-411a-87df-6f50cf8cef90 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.646022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:29.659421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1014/2025] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:29 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:28:29.664991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694e8c4-7954-411a-87df-6f50cf8cef90 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.672953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d694e8c4-7954-411a-87df-6f50cf8cef90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:29.695711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d694e8c4-7954-411a-87df-6f50cf8cef90 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.710181 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c2153-6e96-4f32-b4ee-6b9c698732e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.714154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c2153-6e96-4f32-b4ee-6b9c698732e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.714528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c2153-6e96-4f32-b4ee-6b9c698732e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.714955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c2153-6e96-4f32-b4ee-6b9c698732e2 admin admin] PATH_INFO: `/v3/projects/441ffbe20a794264a493c820b420c250/users/5d40b53d8b37452797f1539b4acadbf4/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.718526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c2153-6e96-4f32-b4ee-6b9c698732e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=441ffbe20a794264a493c820b420c250, user_id=5d40b53d8b37452797f1539b4acadbf4, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.722746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c2153-6e96-4f32-b4ee-6b9c698732e2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.726056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dc864-3834-4b58-9a7a-fc6b29305a2a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:29.740856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:29.743815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.744857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.745609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] PATH_INFO: `/v3/projects/f850eec9d0784fd78f0a59040954c880` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.747627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f850eec9d0784fd78f0a59040954c880)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:29.757951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:29.764104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1015/2026] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:29 2022] PUT /identity/v3/projects/441ffbe20a794264a493c820b420c250/users/5d40b53d8b37452797f1539b4acadbf4/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:29.766530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed18a3-ff31-42bd-8f1f-dfd0f98aab7f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.766530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed18a3-ff31-42bd-8f1f-dfd0f98aab7f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.766530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eed18a3-ff31-42bd-8f1f-dfd0f98aab7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.827130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8eed18a3-ff31-42bd-8f1f-dfd0f98aab7f None None] MFA Rules not processed for user `5d40b53d8b37452797f1539b4acadbf4`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:29.829449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8eed18a3-ff31-42bd-8f1f-dfd0f98aab7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:29.871331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f850eec9d0784fd78f0a59040954c880', 'request_id': 'req-55bf9480-3465-4b1b-965f-0cc549011b4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.887010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f850eec9d0784fd78f0a59040954c880', 'request_id': 'req-55bf9480-3465-4b1b-965f-0cc549011b4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.903808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f850eec9d0784fd78f0a59040954c880', 'request_id': 'req-55bf9480-3465-4b1b-965f-0cc549011b4c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.923241 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] The token cache is being invalidate because project f850eec9d0784fd78f0a59040954c880 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:29.923555 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.924317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-55bf9480-3465-4b1b-965f-0cc549011b4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:29.926240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1012/2027] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:29 2022] DELETE /identity/v3/projects/f850eec9d0784fd78f0a59040954c880 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:29.931563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dac70-f2c3-4087-8f40-88f5246174d6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:29.931563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dac70-f2c3-4087-8f40-88f5246174d6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:29.931805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93dac70-f2c3-4087-8f40-88f5246174d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:29.957231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1016/2028] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:29.957460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c93dac70-f2c3-4087-8f40-88f5246174d6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:29.958727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c93dac70-f2c3-4087-8f40-88f5246174d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:29.969058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed18a3-ff31-42bd-8f1f-dfd0f98aab7f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:29.971824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8eed18a3-ff31-42bd-8f1f-dfd0f98aab7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:29.983826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:30.046052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1013/2029] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:30.054232 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8c647-a987-419d-bbea-96e60e802bd4 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.054526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8c647-a987-419d-bbea-96e60e802bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.054803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8c647-a987-419d-bbea-96e60e802bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.063459 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eed18a3-ff31-42bd-8f1f-dfd0f98aab7f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.073823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b68e3-607d-48f4-a40b-e0d40928cb1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.074726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b68e3-607d-48f4-a40b-e0d40928cb1e service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.075054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b68e3-607d-48f4-a40b-e0d40928cb1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.075386 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b68e3-607d-48f4-a40b-e0d40928cb1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.078451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a04b68e3-607d-48f4-a40b-e0d40928cb1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:30.083891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-36c8c647-a987-419d-bbea-96e60e802bd4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:30.085323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-36c8c647-a987-419d-bbea-96e60e802bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:30.097520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1014/2030] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:30.102142 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8c647-a987-419d-bbea-96e60e802bd4 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.114198 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8c647-a987-419d-bbea-96e60e802bd4 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.120051 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff60683-68b0-4319-9d6d-00119a5ad017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.120737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60683-68b0-4319-9d6d-00119a5ad017 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.121023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60683-68b0-4319-9d6d-00119a5ad017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.121298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff60683-68b0-4319-9d6d-00119a5ad017 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.122096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff60683-68b0-4319-9d6d-00119a5ad017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:30.123257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff60683-68b0-4319-9d6d-00119a5ad017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.124056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff60683-68b0-4319-9d6d-00119a5ad017 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.141900 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1015/2031] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:30.144134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff60683-68b0-4319-9d6d-00119a5ad017 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.152889 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b68e3-607d-48f4-a40b-e0d40928cb1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.154385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b68e3-607d-48f4-a40b-e0d40928cb1e service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.155350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff60683-68b0-4319-9d6d-00119a5ad017 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.161681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe708154-75d5-40ce-a631-54bf4df76ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.162424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe708154-75d5-40ce-a631-54bf4df76ebb admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.162650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe708154-75d5-40ce-a631-54bf4df76ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.162927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe708154-75d5-40ce-a631-54bf4df76ebb admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.163627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe708154-75d5-40ce-a631-54bf4df76ebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.165628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe708154-75d5-40ce-a631-54bf4df76ebb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.166034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1017/2032] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:28:30.201112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1016/2033] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:30.202353 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b68e3-607d-48f4-a40b-e0d40928cb1e service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.217807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b68e3-607d-48f4-a40b-e0d40928cb1e service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.321907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe708154-75d5-40ce-a631-54bf4df76ebb admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.324294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fe708154-75d5-40ce-a631-54bf4df76ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:30.343316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ae28d-8c07-4f9c-a8df-2802d4788aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.344541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ae28d-8c07-4f9c-a8df-2802d4788aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.345001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ae28d-8c07-4f9c-a8df-2802d4788aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.345283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ae28d-8c07-4f9c-a8df-2802d4788aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.347647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ae28d-8c07-4f9c-a8df-2802d4788aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.352850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ae28d-8c07-4f9c-a8df-2802d4788aa0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.364118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:30.406093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1018/2034] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:30.408199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ae28d-8c07-4f9c-a8df-2802d4788aa0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.438381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ae28d-8c07-4f9c-a8df-2802d4788aa0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.453136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2136b4-bbab-47eb-bf72-7977cc2e8771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.454173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2136b4-bbab-47eb-bf72-7977cc2e8771 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.454706 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2136b4-bbab-47eb-bf72-7977cc2e8771 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.455329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2136b4-bbab-47eb-bf72-7977cc2e8771 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.458086 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2136b4-bbab-47eb-bf72-7977cc2e8771 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.462078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2136b4-bbab-47eb-bf72-7977cc2e8771 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.477342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1019/2035] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:30.478899 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2136b4-bbab-47eb-bf72-7977cc2e8771 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.482801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe708154-75d5-40ce-a631-54bf4df76ebb admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.489436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.489986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.490336 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.490678 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] PATH_INFO: `/v3/projects/4f3cdd00ffa446d1b7b79e2376b8e27e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.492043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f3cdd00ffa446d1b7b79e2376b8e27e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.494137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.499561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2136b4-bbab-47eb-bf72-7977cc2e8771 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.507447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcf744-3f42-4ff2-9ae2-71dc73009fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.508215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcf744-3f42-4ff2-9ae2-71dc73009fc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.508702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcf744-3f42-4ff2-9ae2-71dc73009fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.509074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fcf744-3f42-4ff2-9ae2-71dc73009fc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.510263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fcf744-3f42-4ff2-9ae2-71dc73009fc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.512230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fcf744-3f42-4ff2-9ae2-71dc73009fc7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.527128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1020/2036] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:30.531191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcf744-3f42-4ff2-9ae2-71dc73009fc7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.542416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fcf744-3f42-4ff2-9ae2-71dc73009fc7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.584462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f3cdd00ffa446d1b7b79e2376b8e27e', 'request_id': 'req-fdb673f0-5b55-4b67-9e2e-c1f32999a033'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:30.590216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:30.590529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f3cdd00ffa446d1b7b79e2376b8e27e', 'request_id': 'req-fdb673f0-5b55-4b67-9e2e-c1f32999a033'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:30.600357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f3cdd00ffa446d1b7b79e2376b8e27e', 'request_id': 'req-fdb673f0-5b55-4b67-9e2e-c1f32999a033'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:30.615543 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] The token cache is being invalidate because project 4f3cdd00ffa446d1b7b79e2376b8e27e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:30.615918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:30.616594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-fdb673f0-5b55-4b67-9e2e-c1f32999a033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:30.618637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1017/2037] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:30 2022] DELETE /identity/v3/projects/4f3cdd00ffa446d1b7b79e2376b8e27e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:30.623318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700b225-2253-44f1-b986-7dfe57f9047a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.623630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700b225-2253-44f1-b986-7dfe57f9047a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.623895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700b225-2253-44f1-b986-7dfe57f9047a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.651673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7700b225-2253-44f1-b986-7dfe57f9047a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:30.652912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7700b225-2253-44f1-b986-7dfe57f9047a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:30.656733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be367028-51a0-4dfd-8eec-9a4437d5d18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.657948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367028-51a0-4dfd-8eec-9a4437d5d18e admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.658287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367028-51a0-4dfd-8eec-9a4437d5d18e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.658800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be367028-51a0-4dfd-8eec-9a4437d5d18e admin admin] PATH_INFO: `/v3/projects/429f4dad1ff3413bb1f117ae2c50b558/users/fd379feb720845baaebd140a35bc8f6b/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.667903 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1018/2038] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:30.669697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5d801-9719-46d1-abbf-ea55b1c8f507 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.669697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5d801-9719-46d1-abbf-ea55b1c8f507 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.669989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee5d801-9719-46d1-abbf-ea55b1c8f507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.681474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be367028-51a0-4dfd-8eec-9a4437d5d18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=429f4dad1ff3413bb1f117ae2c50b558, user_id=fd379feb720845baaebd140a35bc8f6b, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.685469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be367028-51a0-4dfd-8eec-9a4437d5d18e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.693654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-eee5d801-9719-46d1-abbf-ea55b1c8f507 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:30.693654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-eee5d801-9719-46d1-abbf-ea55b1c8f507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:30.706251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1019/2039] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:30.708572 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1021/2040] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:30 2022] PUT /identity/v3/projects/429f4dad1ff3413bb1f117ae2c50b558/users/fd379feb720845baaebd140a35bc8f6b/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:30.710320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5d801-9719-46d1-abbf-ea55b1c8f507 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.720545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c32075-8664-4867-b604-9b3367203657 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.721452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c32075-8664-4867-b604-9b3367203657 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.721574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee5d801-9719-46d1-abbf-ea55b1c8f507 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.722134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c32075-8664-4867-b604-9b3367203657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.754171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-98c32075-8664-4867-b604-9b3367203657 None None] MFA Rules not processed for user `fd379feb720845baaebd140a35bc8f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:30.755413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-98c32075-8664-4867-b604-9b3367203657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:30.785213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ebfa2-2571-4754-ab16-63d1c4a4fc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.786065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ebfa2-2571-4754-ab16-63d1c4a4fc0c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.786353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ebfa2-2571-4754-ab16-63d1c4a4fc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.786632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8ebfa2-2571-4754-ab16-63d1c4a4fc0c admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.787377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ebfa2-2571-4754-ab16-63d1c4a4fc0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:30.787686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ebfa2-2571-4754-ab16-63d1c4a4fc0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.789295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8ebfa2-2571-4754-ab16-63d1c4a4fc0c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.802455 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1020/2041] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:30.806771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ebfa2-2571-4754-ab16-63d1c4a4fc0c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.817726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8ebfa2-2571-4754-ab16-63d1c4a4fc0c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.822650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd03d7-bd3c-47be-a73e-f8492a9ae1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.823317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd03d7-bd3c-47be-a73e-f8492a9ae1ca admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.823605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd03d7-bd3c-47be-a73e-f8492a9ae1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.823886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd03d7-bd3c-47be-a73e-f8492a9ae1ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.824594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd03d7-bd3c-47be-a73e-f8492a9ae1ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.826687 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd03d7-bd3c-47be-a73e-f8492a9ae1ca admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.828079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1022/2042] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:30.833104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c32075-8664-4867-b604-9b3367203657 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.834682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-98c32075-8664-4867-b604-9b3367203657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:30.849895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c32075-8664-4867-b604-9b3367203657 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.854593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193982a-30a5-4946-be57-76fc12bb7009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.855268 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193982a-30a5-4946-be57-76fc12bb7009 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.855551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193982a-30a5-4946-be57-76fc12bb7009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.855828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193982a-30a5-4946-be57-76fc12bb7009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.856528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193982a-30a5-4946-be57-76fc12bb7009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.858783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193982a-30a5-4946-be57-76fc12bb7009 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.861835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1021/2043] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:30.866387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd03d7-bd3c-47be-a73e-f8492a9ae1ca admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.884814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:30.890252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1023/2044] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:30.895431 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193982a-30a5-4946-be57-76fc12bb7009 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.958394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd03d7-bd3c-47be-a73e-f8492a9ae1ca admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.963466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef72cc-7663-4252-b5fd-9db0bea1e284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.964150 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef72cc-7663-4252-b5fd-9db0bea1e284 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.964432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef72cc-7663-4252-b5fd-9db0bea1e284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.964591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193982a-30a5-4946-be57-76fc12bb7009 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:30.964723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ef72cc-7663-4252-b5fd-9db0bea1e284 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.965415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef72cc-7663-4252-b5fd-9db0bea1e284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.967360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ef72cc-7663-4252-b5fd-9db0bea1e284 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.969650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3d802-c377-4163-ab28-8b25b9224140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:30.970381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3d802-c377-4163-ab28-8b25b9224140 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:30.970825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3d802-c377-4163-ab28-8b25b9224140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:30.971285 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3d802-c377-4163-ab28-8b25b9224140 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:30.972211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3d802-c377-4163-ab28-8b25b9224140 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:30.974396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a3d802-c377-4163-ab28-8b25b9224140 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:30.989986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1022/2045] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/users => generated 294 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:30.990828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef72cc-7663-4252-b5fd-9db0bea1e284 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:30.992457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-57ef72cc-7663-4252-b5fd-9db0bea1e284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:30.998903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1024/2046] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:30 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:30.998903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3d802-c377-4163-ab28-8b25b9224140 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.007680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3d802-c377-4163-ab28-8b25b9224140 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.007957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ef72cc-7663-4252-b5fd-9db0bea1e284 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.012297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9476772-b51f-47a3-a93a-198491a4cd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.012835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.012963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9476772-b51f-47a3-a93a-198491a4cd5f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.013266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9476772-b51f-47a3-a93a-198491a4cd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.013550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9476772-b51f-47a3-a93a-198491a4cd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.013791 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.014076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.014314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9476772-b51f-47a3-a93a-198491a4cd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.014551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] PATH_INFO: `/v3/users/f1af03c329cf4738ab29ddd81851834a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.015584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1af03c329cf4738ab29ddd81851834a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.015897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9476772-b51f-47a3-a93a-198491a4cd5f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.017330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:31.017665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.033629 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1025/2047] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:31.035601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9476772-b51f-47a3-a93a-198491a4cd5f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.046072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9476772-b51f-47a3-a93a-198491a4cd5f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.052315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b9768-0425-40bc-a4dd-859f96c4e2a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.052994 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b9768-0425-40bc-a4dd-859f96c4e2a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.053287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b9768-0425-40bc-a4dd-859f96c4e2a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.053605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b9768-0425-40bc-a4dd-859f96c4e2a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.054466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b9768-0425-40bc-a4dd-859f96c4e2a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.056323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172b9768-0425-40bc-a4dd-859f96c4e2a9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.071126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1026/2048] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:31.071126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b9768-0425-40bc-a4dd-859f96c4e2a9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.073743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1af03c329cf4738ab29ddd81851834a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:31.081222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1af03c329cf4738ab29ddd81851834a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:31.081939 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1af03c329cf4738ab29ddd81851834a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:31.082262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172b9768-0425-40bc-a4dd-859f96c4e2a9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.087166 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529121d8-a05d-487c-bb33-c9fbeeb85f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.087998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529121d8-a05d-487c-bb33-c9fbeeb85f66 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.088293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529121d8-a05d-487c-bb33-c9fbeeb85f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.088570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529121d8-a05d-487c-bb33-c9fbeeb85f66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.089304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529121d8-a05d-487c-bb33-c9fbeeb85f66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.090951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529121d8-a05d-487c-bb33-c9fbeeb85f66 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.094354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1af03c329cf4738ab29ddd81851834a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:31.102984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1af03c329cf4738ab29ddd81851834a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:31.103810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1027/2049] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:31.104583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529121d8-a05d-487c-bb33-c9fbeeb85f66 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.106466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-529121d8-a05d-487c-bb33-c9fbeeb85f66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:31.112407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1023/2050] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:30 2022] DELETE /identity/v3/users/f1af03c329cf4738ab29ddd81851834a => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:31.113974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.115692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:31.123085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529121d8-a05d-487c-bb33-c9fbeeb85f66 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.185415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75770ce-a935-4bd3-b5da-ede99f4e52ac admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.188257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8227ac5-a6e5-4d9b-be7a-33c742a3ff52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.188934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8227ac5-a6e5-4d9b-be7a-33c742a3ff52 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.189219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8227ac5-a6e5-4d9b-be7a-33c742a3ff52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.189531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8227ac5-a6e5-4d9b-be7a-33c742a3ff52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.191017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8227ac5-a6e5-4d9b-be7a-33c742a3ff52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.191987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee1d09-2ce3-498c-aaee-f808448ce6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.192625 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8227ac5-a6e5-4d9b-be7a-33c742a3ff52 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.192903 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee1d09-2ce3-498c-aaee-f808448ce6aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.193200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee1d09-2ce3-498c-aaee-f808448ce6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.193560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee1d09-2ce3-498c-aaee-f808448ce6aa admin admin] PATH_INFO: `/v3/projects/897beed24ab34c7c9c28585980db35bc/users/2070acf5add5412cb2375d9e8296df45/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.206398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1028/2051] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:31.206398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d613385-0903-4f06-812a-b04bfe406901 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.206398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d613385-0903-4f06-812a-b04bfe406901 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.206398 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d613385-0903-4f06-812a-b04bfe406901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.206899 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:31.212951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee1d09-2ce3-498c-aaee-f808448ce6aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=897beed24ab34c7c9c28585980db35bc, user_id=2070acf5add5412cb2375d9e8296df45, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.215891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee1d09-2ce3-498c-aaee-f808448ce6aa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.224352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8d613385-0903-4f06-812a-b04bfe406901 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:31.225724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8d613385-0903-4f06-812a-b04bfe406901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:31.236990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1024/2052] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:31 2022] PUT /identity/v3/projects/897beed24ab34c7c9c28585980db35bc/users/2070acf5add5412cb2375d9e8296df45/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:31.242449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee1d09-2ce3-498c-aaee-f808448ce6aa admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.294369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1029/2053] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:31.295981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e417-921d-4c82-b0eb-de672825815b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.296483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e417-921d-4c82-b0eb-de672825815b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.296709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e417-921d-4c82-b0eb-de672825815b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.318739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee1d09-2ce3-498c-aaee-f808448ce6aa admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.323671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a4d9a-29b1-4120-8bee-56e9d5af85a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.325267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a4d9a-29b1-4120-8bee-56e9d5af85a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.325267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a4d9a-29b1-4120-8bee-56e9d5af85a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.325267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a4d9a-29b1-4120-8bee-56e9d5af85a7 admin admin] PATH_INFO: `/v3/projects/169a45fb4c7b41ee8b28f84ede0a33a1/users/23a9fb25e87c453db58599824468e0b8/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.328201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a1a0e417-921d-4c82-b0eb-de672825815b None None] MFA Rules not processed for user `2070acf5add5412cb2375d9e8296df45`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:31.329669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a1a0e417-921d-4c82-b0eb-de672825815b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:31.346222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a4d9a-29b1-4120-8bee-56e9d5af85a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=169a45fb4c7b41ee8b28f84ede0a33a1, user_id=23a9fb25e87c453db58599824468e0b8, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.349054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a4d9a-29b1-4120-8bee-56e9d5af85a7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.365135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1025/2054] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:31 2022] PUT /identity/v3/projects/169a45fb4c7b41ee8b28f84ede0a33a1/users/23a9fb25e87c453db58599824468e0b8/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:31.366195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a4d9a-29b1-4120-8bee-56e9d5af85a7 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.382813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-832a4d9a-29b1-4120-8bee-56e9d5af85a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:31.481063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a4d9a-29b1-4120-8bee-56e9d5af85a7 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.485720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d79a51-fc1f-4e4b-b641-e4542f5cca6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.486392 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d79a51-fc1f-4e4b-b641-e4542f5cca6c service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.486913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d79a51-fc1f-4e4b-b641-e4542f5cca6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.487234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d79a51-fc1f-4e4b-b641-e4542f5cca6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.500159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:31.506826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1030/2055] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:31.508189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c071d4-5e6a-4831-972a-125dd19e011c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.508454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c071d4-5e6a-4831-972a-125dd19e011c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.508737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c071d4-5e6a-4831-972a-125dd19e011c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.575140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-88c071d4-5e6a-4831-972a-125dd19e011c None None] MFA Rules not processed for user `23a9fb25e87c453db58599824468e0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:31.576490 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-88c071d4-5e6a-4831-972a-125dd19e011c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:31.651854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d79a51-fc1f-4e4b-b641-e4542f5cca6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.655448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d79a51-fc1f-4e4b-b641-e4542f5cca6c service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.676936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1026/2056] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:31.678965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d79a51-fc1f-4e4b-b641-e4542f5cca6c service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.697440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d79a51-fc1f-4e4b-b641-e4542f5cca6c service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.737737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75819cca-8b3f-4d6c-9207-6fd2c4c74444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.738565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75819cca-8b3f-4d6c-9207-6fd2c4c74444 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.738858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75819cca-8b3f-4d6c-9207-6fd2c4c74444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.739138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75819cca-8b3f-4d6c-9207-6fd2c4c74444 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.739851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75819cca-8b3f-4d6c-9207-6fd2c4c74444 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.743117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75819cca-8b3f-4d6c-9207-6fd2c4c74444 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.757475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1031/2057] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:31.765638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c071d4-5e6a-4831-972a-125dd19e011c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.786628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c071d4-5e6a-4831-972a-125dd19e011c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:31.792610 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d3203-83dd-45f0-9b5f-1d21cca38e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:31.793305 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d3203-83dd-45f0-9b5f-1d21cca38e5e admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:31.793618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d3203-83dd-45f0-9b5f-1d21cca38e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:31.793900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452d3203-83dd-45f0-9b5f-1d21cca38e5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:31.794752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d3203-83dd-45f0-9b5f-1d21cca38e5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:31.801840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:31.813964 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1027/2058] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:31.814052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452d3203-83dd-45f0-9b5f-1d21cca38e5e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:31.815710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75819cca-8b3f-4d6c-9207-6fd2c4c74444 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.872953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1032/2059] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:31.876523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d3203-83dd-45f0-9b5f-1d21cca38e5e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:31.960690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75819cca-8b3f-4d6c-9207-6fd2c4c74444 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.036044 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452d3203-83dd-45f0-9b5f-1d21cca38e5e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.046027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda117e7-4f13-4599-9d51-3add97f0bd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.047769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda117e7-4f13-4599-9d51-3add97f0bd3e admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.048054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda117e7-4f13-4599-9d51-3add97f0bd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.048331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda117e7-4f13-4599-9d51-3add97f0bd3e admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.054003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda117e7-4f13-4599-9d51-3add97f0bd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.060456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda117e7-4f13-4599-9d51-3add97f0bd3e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.109789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1033/2060] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/users => generated 295 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:32.111160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744cc73-3c60-4ee9-a859-9b39354be9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.112406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744cc73-3c60-4ee9-a859-9b39354be9eb admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.114207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda117e7-4f13-4599-9d51-3add97f0bd3e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.114827 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744cc73-3c60-4ee9-a859-9b39354be9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.117914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744cc73-3c60-4ee9-a859-9b39354be9eb admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.119635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744cc73-3c60-4ee9-a859-9b39354be9eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.125684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c744cc73-3c60-4ee9-a859-9b39354be9eb admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.144908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda117e7-4f13-4599-9d51-3add97f0bd3e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.156294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381d7ca-06ca-46e1-9926-ba4a31987223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.157024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381d7ca-06ca-46e1-9926-ba4a31987223 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.157310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381d7ca-06ca-46e1-9926-ba4a31987223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.157620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381d7ca-06ca-46e1-9926-ba4a31987223 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.159228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381d7ca-06ca-46e1-9926-ba4a31987223 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.168128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381d7ca-06ca-46e1-9926-ba4a31987223 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.180244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1028/2061] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:31 2022] POST /identity/v3/users => generated 299 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:32.184885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744cc73-3c60-4ee9-a859-9b39354be9eb admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.197431 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1034/2062] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:32.200389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381d7ca-06ca-46e1-9926-ba4a31987223 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.207375 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744cc73-3c60-4ee9-a859-9b39354be9eb admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.223399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8df1c-67f7-4f85-baf2-2efdf2bb6a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.230980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381d7ca-06ca-46e1-9926-ba4a31987223 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.238309 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8df1c-67f7-4f85-baf2-2efdf2bb6a86 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.238824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8df1c-67f7-4f85-baf2-2efdf2bb6a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.239067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8df1c-67f7-4f85-baf2-2efdf2bb6a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.239981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8df1c-67f7-4f85-baf2-2efdf2bb6a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.242150 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8df1c-67f7-4f85-baf2-2efdf2bb6a86 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.244482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3618182-4f81-406f-9e3d-6b27be1a3509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.245414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3618182-4f81-406f-9e3d-6b27be1a3509 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.245712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3618182-4f81-406f-9e3d-6b27be1a3509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.245999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3618182-4f81-406f-9e3d-6b27be1a3509 admin admin] PATH_INFO: `/v3/projects/7d6ef193a7c641d19b4425e91f781c00/users/cdeed567c36f4077ad2bc531afa80897/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.271211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1029/2063] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:32.273095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8df1c-67f7-4f85-baf2-2efdf2bb6a86 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.280056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3618182-4f81-406f-9e3d-6b27be1a3509 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6ef193a7c641d19b4425e91f781c00, user_id=cdeed567c36f4077ad2bc531afa80897, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.283724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3618182-4f81-406f-9e3d-6b27be1a3509 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.289416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8df1c-67f7-4f85-baf2-2efdf2bb6a86 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.297850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223fd78-b80d-41d7-b344-4d95e17f5b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.298771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223fd78-b80d-41d7-b344-4d95e17f5b8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.299096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223fd78-b80d-41d7-b344-4d95e17f5b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.299376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223fd78-b80d-41d7-b344-4d95e17f5b8f admin admin] PATH_INFO: `/v3/projects/caab519bb0c74fd6a0cbe7e4dbf89f17/users/bb1f5f228747415d90d6f73154cbef79/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.304223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1035/2064] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:32 2022] PUT /identity/v3/projects/7d6ef193a7c641d19b4425e91f781c00/users/cdeed567c36f4077ad2bc531afa80897/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:32.307954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3618182-4f81-406f-9e3d-6b27be1a3509 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.337280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223fd78-b80d-41d7-b344-4d95e17f5b8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=caab519bb0c74fd6a0cbe7e4dbf89f17, user_id=bb1f5f228747415d90d6f73154cbef79, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.342230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223fd78-b80d-41d7-b344-4d95e17f5b8f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.376113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:32.381996 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1030/2065] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:32 2022] PUT /identity/v3/projects/caab519bb0c74fd6a0cbe7e4dbf89f17/users/bb1f5f228747415d90d6f73154cbef79/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:32.385604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223fd78-b80d-41d7-b344-4d95e17f5b8f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.487688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223fd78-b80d-41d7-b344-4d95e17f5b8f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.515314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3618182-4f81-406f-9e3d-6b27be1a3509 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.516381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10e015-5a31-4f0d-9e22-41704f4596ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.517140 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10e015-5a31-4f0d-9e22-41704f4596ba admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.518448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10e015-5a31-4f0d-9e22-41704f4596ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.525835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b10e015-5a31-4f0d-9e22-41704f4596ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.526664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10e015-5a31-4f0d-9e22-41704f4596ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.528300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b10e015-5a31-4f0d-9e22-41704f4596ba admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.534584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ac502-974d-4a40-b6a8-61620980300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.536404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ac502-974d-4a40-b6a8-61620980300b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.536855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ac502-974d-4a40-b6a8-61620980300b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.537321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:32.538365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ac502-974d-4a40-b6a8-61620980300b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.539096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ac502-974d-4a40-b6a8-61620980300b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.541133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ac502-974d-4a40-b6a8-61620980300b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.556577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1036/2066] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:28:32.561786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1031/2067] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:28:32.563040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10e015-5a31-4f0d-9e22-41704f4596ba admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.563801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ac502-974d-4a40-b6a8-61620980300b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.576559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b10e015-5a31-4f0d-9e22-41704f4596ba admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.584801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6ccb-2b32-45e7-9bfa-999b75954bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.585835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6ccb-2b32-45e7-9bfa-999b75954bc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.586112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6ccb-2b32-45e7-9bfa-999b75954bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.586390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6ccb-2b32-45e7-9bfa-999b75954bc2 admin admin] PATH_INFO: `/v3/projects/caab519bb0c74fd6a0cbe7e4dbf89f17/users/bb1f5f228747415d90d6f73154cbef79/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.589091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ac502-974d-4a40-b6a8-61620980300b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.594531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc130b6-6edb-463f-9dc1-febbb090bf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.596815 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc130b6-6edb-463f-9dc1-febbb090bf84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.597117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc130b6-6edb-463f-9dc1-febbb090bf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.597652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc130b6-6edb-463f-9dc1-febbb090bf84 admin admin] PATH_INFO: `/v3/projects/7d6ef193a7c641d19b4425e91f781c00/users/cdeed567c36f4077ad2bc531afa80897/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.599007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc6ccb-2b32-45e7-9bfa-999b75954bc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=caab519bb0c74fd6a0cbe7e4dbf89f17, user_id=bb1f5f228747415d90d6f73154cbef79, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.601269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc130b6-6edb-463f-9dc1-febbb090bf84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d6ef193a7c641d19b4425e91f781c00, user_id=cdeed567c36f4077ad2bc531afa80897, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.602077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc6ccb-2b32-45e7-9bfa-999b75954bc2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.604142 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc130b6-6edb-463f-9dc1-febbb090bf84 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.617407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1032/2068] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:32 2022] PUT /identity/v3/projects/caab519bb0c74fd6a0cbe7e4dbf89f17/users/bb1f5f228747415d90d6f73154cbef79/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:32.629781 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b0818-a0e9-4858-a736-33895be02229 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.630524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b0818-a0e9-4858-a736-33895be02229 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.630653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0b0818-a0e9-4858-a736-33895be02229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.637718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1037/2069] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:32 2022] PUT /identity/v3/projects/7d6ef193a7c641d19b4425e91f781c00/users/cdeed567c36f4077ad2bc531afa80897/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:32.648761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcf13b-0dee-4049-aed2-c3244160202a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.649182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcf13b-0dee-4049-aed2-c3244160202a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.649863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfcf13b-0dee-4049-aed2-c3244160202a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.683819 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4a0b0818-a0e9-4858-a736-33895be02229 None None] MFA Rules not processed for user `bb1f5f228747415d90d6f73154cbef79`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:32.686301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4a0b0818-a0e9-4858-a736-33895be02229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:32.708463 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6bfcf13b-0dee-4049-aed2-c3244160202a None None] MFA Rules not processed for user `cdeed567c36f4077ad2bc531afa80897`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:32.710240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6bfcf13b-0dee-4049-aed2-c3244160202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:32.772204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1033/2070] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:32 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:32.773593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b0818-a0e9-4858-a736-33895be02229 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.774697 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4a0b0818-a0e9-4858-a736-33895be02229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:32.792916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1038/2071] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:32.794252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcf13b-0dee-4049-aed2-c3244160202a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.797006 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6bfcf13b-0dee-4049-aed2-c3244160202a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:32.844454 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0b0818-a0e9-4858-a736-33895be02229 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.850618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.851853 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.852212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.852575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] PATH_INFO: `/v3/projects/9fa24df0f2304145aafafbd4c641111f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.853956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9fa24df0f2304145aafafbd4c641111f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.856084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.862739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfcf13b-0dee-4049-aed2-c3244160202a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.866959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e4718-1afc-4aa4-aa0f-06ba70363128 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.867596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e4718-1afc-4aa4-aa0f-06ba70363128 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.867882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e4718-1afc-4aa4-aa0f-06ba70363128 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.868156 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08e4718-1afc-4aa4-aa0f-06ba70363128 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.878641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e4718-1afc-4aa4-aa0f-06ba70363128 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:32.880102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08e4718-1afc-4aa4-aa0f-06ba70363128 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:32.891366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1039/2072] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:32.892504 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e4718-1afc-4aa4-aa0f-06ba70363128 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.894389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e08e4718-1afc-4aa4-aa0f-06ba70363128 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:32.906852 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08e4718-1afc-4aa4-aa0f-06ba70363128 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:32.918537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9fa24df0f2304145aafafbd4c641111f', 'request_id': 'req-4d41e486-6239-4c50-ac69-b8b26b462ecc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:32.925231 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9fa24df0f2304145aafafbd4c641111f', 'request_id': 'req-4d41e486-6239-4c50-ac69-b8b26b462ecc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:32.937840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9fa24df0f2304145aafafbd4c641111f', 'request_id': 'req-4d41e486-6239-4c50-ac69-b8b26b462ecc'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:32.949586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:32.953427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:32.955423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] The token cache is being invalidate because project 9fa24df0f2304145aafafbd4c641111f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:32.955958 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:32.957396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:32.962554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1034/2073] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:32 2022] DELETE /identity/v3/projects/9fa24df0f2304145aafafbd4c641111f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:32.966493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:32.973541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:32.984972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:32.985932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:32.986390 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:32.987397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:32.991401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:33.086077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d41e486-6239-4c50-ac69-b8b26b462ecc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:33.086511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:33.089244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:33.099701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:33.100802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:33.101658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:33.102109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] PATH_INFO: `/v3/users/33d1d99271654ef08acc835b15e46084` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:33.103192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1040/2074] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:33.108954 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d1d99271654ef08acc835b15e46084)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:33.110977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:33.152191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:33.153733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:33.166902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5bb7d-6dc3-47cf-b969-f59655156250 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:33.172091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d3945-1ba4-47e3-9511-192cdc6a2fec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:33.172982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d3945-1ba4-47e3-9511-192cdc6a2fec service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:33.173338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d3945-1ba4-47e3-9511-192cdc6a2fec service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:33.173636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d3945-1ba4-47e3-9511-192cdc6a2fec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:33.181519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-211d3945-1ba4-47e3-9511-192cdc6a2fec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:33.186031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d1d99271654ef08acc835b15e46084'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:33.197738 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d1d99271654ef08acc835b15e46084'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:33.198724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d1d99271654ef08acc835b15e46084'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:33.216523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d1d99271654ef08acc835b15e46084'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:33.238780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d1d99271654ef08acc835b15e46084'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:33.254280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1035/2075] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:32 2022] DELETE /identity/v3/users/33d1d99271654ef08acc835b15e46084 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:33.272715 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d3945-1ba4-47e3-9511-192cdc6a2fec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:33.274284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211d3945-1ba4-47e3-9511-192cdc6a2fec service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:33.285463 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1041/2076] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:33 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:33.739406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:33.741101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:33.809995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d3945-1ba4-47e3-9511-192cdc6a2fec service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:33.829352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d3945-1ba4-47e3-9511-192cdc6a2fec service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:33.925199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccbee0d-9aae-43ab-8b62-03f6421e2a5a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:33.930644 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:33.931665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:33.932526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:33.932526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] PATH_INFO: `/v3/users/2070acf5add5412cb2375d9e8296df45` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:33.933798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2070acf5add5412cb2375d9e8296df45)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:33.936270 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:33.961157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:33.962975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:33.963616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:33.964997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] PATH_INFO: `/v3/projects/eebdcb47c1c24269bb72643cbaf06410` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:33.968014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eebdcb47c1c24269bb72643cbaf06410)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:33.976705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:33.986339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:34.022566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2070acf5add5412cb2375d9e8296df45'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.032408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2070acf5add5412cb2375d9e8296df45'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.033304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2070acf5add5412cb2375d9e8296df45'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.048681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:34.052382 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2070acf5add5412cb2375d9e8296df45'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.064402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ea3048f5-710a-46a5-a83c-9967de5d5406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2070acf5add5412cb2375d9e8296df45'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.075600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1036/2077] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:33 2022] DELETE /identity/v3/users/2070acf5add5412cb2375d9e8296df45 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:34.082465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eebdcb47c1c24269bb72643cbaf06410', 'request_id': 'req-19e75408-d746-42ec-8d68-e125075047a4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.083468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8572684-4d39-43ba-86f3-57ddad4d0daf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.083951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8572684-4d39-43ba-86f3-57ddad4d0daf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.084466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8572684-4d39-43ba-86f3-57ddad4d0daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.094687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eebdcb47c1c24269bb72643cbaf06410', 'request_id': 'req-19e75408-d746-42ec-8d68-e125075047a4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.108188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eebdcb47c1c24269bb72643cbaf06410', 'request_id': 'req-19e75408-d746-42ec-8d68-e125075047a4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.116298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d8572684-4d39-43ba-86f3-57ddad4d0daf None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:34.118736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d8572684-4d39-43ba-86f3-57ddad4d0daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:34.123535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] The token cache is being invalidate because project eebdcb47c1c24269bb72643cbaf06410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:34.123867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.124556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.129456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1042/2078] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:33 2022] DELETE /identity/v3/projects/eebdcb47c1c24269bb72643cbaf06410 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:34.133974 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.135545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:34.213768 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1037/2079] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:34.225842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8572684-4d39-43ba-86f3-57ddad4d0daf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.228567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d8572684-4d39-43ba-86f3-57ddad4d0daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:34.283855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e75408-d746-42ec-8d68-e125075047a4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:34.290019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:34.290987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.291385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.291767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] PATH_INFO: `/v3/users/4d5c2a75614b47e6b54c58e2a9cf5d30` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.296452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d5c2a75614b47e6b54c58e2a9cf5d30)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:34.299573 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:34.340213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8572684-4d39-43ba-86f3-57ddad4d0daf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:34.358829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5830f-f1ef-4efa-8db0-2b3885baf645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:34.359614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5830f-f1ef-4efa-8db0-2b3885baf645 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.359943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5830f-f1ef-4efa-8db0-2b3885baf645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.361114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5830f-f1ef-4efa-8db0-2b3885baf645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.373569 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5830f-f1ef-4efa-8db0-2b3885baf645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:34.375118 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5830f-f1ef-4efa-8db0-2b3885baf645 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:34.384192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d5c2a75614b47e6b54c58e2a9cf5d30'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.390713 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1038/2080] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:34.392396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d5c2a75614b47e6b54c58e2a9cf5d30'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.393840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d5c2a75614b47e6b54c58e2a9cf5d30'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.405878 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d5c2a75614b47e6b54c58e2a9cf5d30'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.427379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d5c2a75614b47e6b54c58e2a9cf5d30'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.436306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1043/2081] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:34 2022] DELETE /identity/v3/users/4d5c2a75614b47e6b54c58e2a9cf5d30 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:34.764434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5830f-f1ef-4efa-8db0-2b3885baf645 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.766042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-43b5830f-f1ef-4efa-8db0-2b3885baf645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:34.781419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5830f-f1ef-4efa-8db0-2b3885baf645 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:34.828666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:34.831384 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:34.855547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:34.884285 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:34.899044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:34.900192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.900822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.901121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] PATH_INFO: `/v3/projects/897beed24ab34c7c9c28585980db35bc` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.903077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=897beed24ab34c7c9c28585980db35bc)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:34.905679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:34.933275 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067cc76-00c1-4bd0-b18a-91c97add8534 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:34.942887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:34.945750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:34.948022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:34.948022 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] PATH_INFO: `/v3/projects/ff1afcf633e24697bdf5e630092d24c3` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:34.951037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff1afcf633e24697bdf5e630092d24c3)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:34.952193 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:34.973965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '897beed24ab34c7c9c28585980db35bc', 'request_id': 'req-7774e51e-2665-4b02-971f-9600d2cba5ce'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:34.991327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '897beed24ab34c7c9c28585980db35bc', 'request_id': 'req-7774e51e-2665-4b02-971f-9600d2cba5ce'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.016895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '897beed24ab34c7c9c28585980db35bc', 'request_id': 'req-7774e51e-2665-4b02-971f-9600d2cba5ce'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.055040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] The token cache is being invalidate because project 897beed24ab34c7c9c28585980db35bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:35.055040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.055040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.055040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1039/2082] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:34 2022] DELETE /identity/v3/projects/897beed24ab34c7c9c28585980db35bc => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:35.060172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff1afcf633e24697bdf5e630092d24c3', 'request_id': 'req-7610d47d-dfe0-4529-bd00-c0daee4741d4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.069831 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.071794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:35.074513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff1afcf633e24697bdf5e630092d24c3', 'request_id': 'req-7610d47d-dfe0-4529-bd00-c0daee4741d4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.090174 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff1afcf633e24697bdf5e630092d24c3', 'request_id': 'req-7610d47d-dfe0-4529-bd00-c0daee4741d4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.136131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] The token cache is being invalidate because project ff1afcf633e24697bdf5e630092d24c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:35.136608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.137709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7610d47d-dfe0-4529-bd00-c0daee4741d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.141010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1044/2083] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:34 2022] DELETE /identity/v3/projects/ff1afcf633e24697bdf5e630092d24c3 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:35.151908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4deb54-6d78-4e26-a05c-207e651f289d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.151908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4deb54-6d78-4e26-a05c-207e651f289d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.151908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4deb54-6d78-4e26-a05c-207e651f289d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.184568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9a4deb54-6d78-4e26-a05c-207e651f289d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:35.186112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9a4deb54-6d78-4e26-a05c-207e651f289d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:35.273290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7774e51e-2665-4b02-971f-9600d2cba5ce admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.387349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:35.388250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.388954 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.389551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] PATH_INFO: `/v3/users/bb1f5f228747415d90d6f73154cbef79` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.399462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb1f5f228747415d90d6f73154cbef79)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:35.399971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1045/2084] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:35.400302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:35.402357 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317dc5b-cc83-4b12-9feb-f176ad67db9b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.402865 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317dc5b-cc83-4b12-9feb-f176ad67db9b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.403359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3317dc5b-cc83-4b12-9feb-f176ad67db9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.455041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:35.492330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3317dc5b-cc83-4b12-9feb-f176ad67db9b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:35.503424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3317dc5b-cc83-4b12-9feb-f176ad67db9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:35.528765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb1f5f228747415d90d6f73154cbef79'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.543311 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1046/2085] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:35.550828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317dc5b-cc83-4b12-9feb-f176ad67db9b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.560518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb1f5f228747415d90d6f73154cbef79'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.562010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb1f5f228747415d90d6f73154cbef79'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.573737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3317dc5b-cc83-4b12-9feb-f176ad67db9b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.581968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca62054-0014-4d5c-b5ee-a90d5b9072a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:35.582854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca62054-0014-4d5c-b5ee-a90d5b9072a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:35.583325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca62054-0014-4d5c-b5ee-a90d5b9072a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:35.583676 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca62054-0014-4d5c-b5ee-a90d5b9072a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:35.584516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca62054-0014-4d5c-b5ee-a90d5b9072a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:35.584934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca62054-0014-4d5c-b5ee-a90d5b9072a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:35.586836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca62054-0014-4d5c-b5ee-a90d5b9072a7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:35.601461 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb1f5f228747415d90d6f73154cbef79'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.616200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1047/2086] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:35.632196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca62054-0014-4d5c-b5ee-a90d5b9072a7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.775215 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb1f5f228747415d90d6f73154cbef79'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:35.786180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca62054-0014-4d5c-b5ee-a90d5b9072a7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:35.793564 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1040/2087] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:35 2022] DELETE /identity/v3/users/bb1f5f228747415d90d6f73154cbef79 => generated 0 bytes in 734 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:35.795361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:35.798057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:35.964069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:36.032222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a671b-ca7e-48a4-907d-226506da19ad admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.044487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889bbfe-98a9-4e17-9e0c-d276d798932d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.045476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89e09b-1a4f-4651-ac31-9a0e711376a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.049132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889bbfe-98a9-4e17-9e0c-d276d798932d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.049737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889bbfe-98a9-4e17-9e0c-d276d798932d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.050166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d889bbfe-98a9-4e17-9e0c-d276d798932d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.054252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89e09b-1a4f-4651-ac31-9a0e711376a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.054886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89e09b-1a4f-4651-ac31-9a0e711376a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.055396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d889bbfe-98a9-4e17-9e0c-d276d798932d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:36.059855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89e09b-1a4f-4651-ac31-9a0e711376a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.067892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89e09b-1a4f-4651-ac31-9a0e711376a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.078080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df89e09b-1a4f-4651-ac31-9a0e711376a0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.175902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1048/2088] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:35 2022] POST /identity/v3/projects => generated 385 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:36.176989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89e09b-1a4f-4651-ac31-9a0e711376a0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:36.184270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-df89e09b-1a4f-4651-ac31-9a0e711376a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:36.242763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889bbfe-98a9-4e17-9e0c-d276d798932d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.244531 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d889bbfe-98a9-4e17-9e0c-d276d798932d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.256783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1041/2089] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:35 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:36.337010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89e09b-1a4f-4651-ac31-9a0e711376a0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.346948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edeca2-b99e-423a-8acb-d38abc2bf433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.350764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edeca2-b99e-423a-8acb-d38abc2bf433 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.351094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edeca2-b99e-423a-8acb-d38abc2bf433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.351460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edeca2-b99e-423a-8acb-d38abc2bf433 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.352631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edeca2-b99e-423a-8acb-d38abc2bf433 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.355981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edeca2-b99e-423a-8acb-d38abc2bf433 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.436618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1049/2090] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:36 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:36.442307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889bbfe-98a9-4e17-9e0c-d276d798932d service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:36.458210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d889bbfe-98a9-4e17-9e0c-d276d798932d service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.467313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf841f-54e9-419e-b4f3-68589cf6b45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.469571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf841f-54e9-419e-b4f3-68589cf6b45e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.469937 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf841f-54e9-419e-b4f3-68589cf6b45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.470423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdf841f-54e9-419e-b4f3-68589cf6b45e admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.473256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf841f-54e9-419e-b4f3-68589cf6b45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.473809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdf841f-54e9-419e-b4f3-68589cf6b45e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.494361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edeca2-b99e-423a-8acb-d38abc2bf433 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:36.494839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1042/2091] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:28:36.504541 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f0edeca2-b99e-423a-8acb-d38abc2bf433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:36.507924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf841f-54e9-419e-b4f3-68589cf6b45e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:36.522128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdf841f-54e9-419e-b4f3-68589cf6b45e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.535022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99e692-8a3d-4ca7-8381-6e09c0103ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.536961 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99e692-8a3d-4ca7-8381-6e09c0103ab4 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.537296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99e692-8a3d-4ca7-8381-6e09c0103ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.538818 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb99e692-8a3d-4ca7-8381-6e09c0103ab4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.539598 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99e692-8a3d-4ca7-8381-6e09c0103ab4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.544494 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb99e692-8a3d-4ca7-8381-6e09c0103ab4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.571196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1043/2092] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:36 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:36.575741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99e692-8a3d-4ca7-8381-6e09c0103ab4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:36.590278 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb99e692-8a3d-4ca7-8381-6e09c0103ab4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.594886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edeca2-b99e-423a-8acb-d38abc2bf433 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:36.599036 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cbc113-08b2-48c3-ae3f-f381564c9045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.600524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbc113-08b2-48c3-ae3f-f381564c9045 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.600903 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbc113-08b2-48c3-ae3f-f381564c9045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.601351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cbc113-08b2-48c3-ae3f-f381564c9045 admin admin] PATH_INFO: `/v3/projects/8d10f59a27974c319846702d6b9a6944/users/a409bd6c4dc24a15ace28d58282c41bb/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.620544 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:36.629273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.630542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.634378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] PATH_INFO: `/v3/projects/caab519bb0c74fd6a0cbe7e4dbf89f17` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.648635 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] RBAC: Authorizing `identity:delete_project(project_id=caab519bb0c74fd6a0cbe7e4dbf89f17)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.665646 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.670160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cbc113-08b2-48c3-ae3f-f381564c9045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d10f59a27974c319846702d6b9a6944, user_id=a409bd6c4dc24a15ace28d58282c41bb, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:36.687845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cbc113-08b2-48c3-ae3f-f381564c9045 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:36.752616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1044/2093] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:36 2022] PUT /identity/v3/projects/8d10f59a27974c319846702d6b9a6944/users/a409bd6c4dc24a15ace28d58282c41bb/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:36.754897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f91038-bb07-473e-88ab-fc5ac9c795c5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.758348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f91038-bb07-473e-88ab-fc5ac9c795c5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.758861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f91038-bb07-473e-88ab-fc5ac9c795c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.771672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:36.836249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'caab519bb0c74fd6a0cbe7e4dbf89f17', 'request_id': 'req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:36.845101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-91f91038-bb07-473e-88ab-fc5ac9c795c5 None None] MFA Rules not processed for user `a409bd6c4dc24a15ace28d58282c41bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:36.848191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-91f91038-bb07-473e-88ab-fc5ac9c795c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:36.854425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'caab519bb0c74fd6a0cbe7e4dbf89f17', 'request_id': 'req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:36.874505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'caab519bb0c74fd6a0cbe7e4dbf89f17', 'request_id': 'req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:36.921323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] The token cache is being invalidate because project caab519bb0c74fd6a0cbe7e4dbf89f17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:36.922829 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:36.927654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05e5ea9b-74b6-4fde-97cc-f3eb584ac49a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:36.932320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1050/2094] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:36 2022] DELETE /identity/v3/projects/caab519bb0c74fd6a0cbe7e4dbf89f17 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:36.946639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00c16-f64c-4918-b945-1944bbe0af03 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:36.946941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00c16-f64c-4918-b945-1944bbe0af03 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:36.947231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00c16-f64c-4918-b945-1944bbe0af03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:36.995673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-43a00c16-f64c-4918-b945-1944bbe0af03 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:36.997498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-43a00c16-f64c-4918-b945-1944bbe0af03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.108104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:37.160868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1045/2095] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:36 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:37.170107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f91038-bb07-473e-88ab-fc5ac9c795c5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.175032 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-91f91038-bb07-473e-88ab-fc5ac9c795c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.178811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1051/2096] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:37.188620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483b66a-b647-4e4b-8421-9e749df43880 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.189971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483b66a-b647-4e4b-8421-9e749df43880 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.193084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e483b66a-b647-4e4b-8421-9e749df43880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.245017 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f91038-bb07-473e-88ab-fc5ac9c795c5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.257412 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e483b66a-b647-4e4b-8421-9e749df43880 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:37.259211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e483b66a-b647-4e4b-8421-9e749df43880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.278433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375200e6-927e-4826-bb24-fd47de5878b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.279880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375200e6-927e-4826-bb24-fd47de5878b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.279997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375200e6-927e-4826-bb24-fd47de5878b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.288080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375200e6-927e-4826-bb24-fd47de5878b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.289337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375200e6-927e-4826-bb24-fd47de5878b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.296369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375200e6-927e-4826-bb24-fd47de5878b1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.301814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1052/2097] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:37.306083 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483b66a-b647-4e4b-8421-9e749df43880 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.308518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e483b66a-b647-4e4b-8421-9e749df43880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.336942 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e483b66a-b647-4e4b-8421-9e749df43880 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.347376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.348419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.348721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.349007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] PATH_INFO: `/v3/users/fd379feb720845baaebd140a35bc8f6b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.350371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd379feb720845baaebd140a35bc8f6b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.352622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.354833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1046/2098] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:37 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:37.358645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375200e6-927e-4826-bb24-fd47de5878b1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.435873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd379feb720845baaebd140a35bc8f6b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:37.451278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd379feb720845baaebd140a35bc8f6b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:37.452145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd379feb720845baaebd140a35bc8f6b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:37.467693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd379feb720845baaebd140a35bc8f6b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:37.482386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375200e6-927e-4826-bb24-fd47de5878b1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.498278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd379feb720845baaebd140a35bc8f6b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:37.502929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8638dd5-8c3d-4f65-a54f-0c4c08c70afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.503529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8638dd5-8c3d-4f65-a54f-0c4c08c70afd admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.503802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8638dd5-8c3d-4f65-a54f-0c4c08c70afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.504086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8638dd5-8c3d-4f65-a54f-0c4c08c70afd admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.504843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8638dd5-8c3d-4f65-a54f-0c4c08c70afd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:37.505206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8638dd5-8c3d-4f65-a54f-0c4c08c70afd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.506293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1053/2099] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:37 2022] DELETE /identity/v3/users/fd379feb720845baaebd140a35bc8f6b => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:37.506946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.508820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.512280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8638dd5-8c3d-4f65-a54f-0c4c08c70afd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.542548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1047/2100] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:37.544374 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9014ed-8229-472f-bdfd-cef98b701b66 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.544549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9014ed-8229-472f-bdfd-cef98b701b66 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.544868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9014ed-8229-472f-bdfd-cef98b701b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.574193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0e9014ed-8229-472f-bdfd-cef98b701b66 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:37.575837 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0e9014ed-8229-472f-bdfd-cef98b701b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.613043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146ffe46-f05c-429f-9c1d-e568b7a7b84a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.619609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07e7c3-6c6f-4b6c-8051-dc62dad2073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.620616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07e7c3-6c6f-4b6c-8051-dc62dad2073c admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.621086 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07e7c3-6c6f-4b6c-8051-dc62dad2073c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.621658 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd07e7c3-6c6f-4b6c-8051-dc62dad2073c admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.622568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07e7c3-6c6f-4b6c-8051-dc62dad2073c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.624775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd07e7c3-6c6f-4b6c-8051-dc62dad2073c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.655040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1054/2101] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:37 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:37.655040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07e7c3-6c6f-4b6c-8051-dc62dad2073c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.655040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dd07e7c3-6c6f-4b6c-8051-dc62dad2073c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.668847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1048/2102] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:37.668847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9014ed-8229-472f-bdfd-cef98b701b66 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.675686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:37.678417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd07e7c3-6c6f-4b6c-8051-dc62dad2073c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.684360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9014ed-8229-472f-bdfd-cef98b701b66 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.689503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6ee2f-0172-4d54-8871-f5e0582c503b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.690623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6ee2f-0172-4d54-8871-f5e0582c503b admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.691033 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6ee2f-0172-4d54-8871-f5e0582c503b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.691675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b6ee2f-0172-4d54-8871-f5e0582c503b admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.692420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6ee2f-0172-4d54-8871-f5e0582c503b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.694526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f0e6f-c593-4476-8aec-c78251f24a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.694805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b6ee2f-0172-4d54-8871-f5e0582c503b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.695657 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f0e6f-c593-4476-8aec-c78251f24a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.696152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f0e6f-c593-4476-8aec-c78251f24a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.696645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338f0e6f-c593-4476-8aec-c78251f24a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.698077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f0e6f-c593-4476-8aec-c78251f24a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.701763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338f0e6f-c593-4476-8aec-c78251f24a10 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.730483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1049/2103] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:37.730483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f0e6f-c593-4476-8aec-c78251f24a10 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.731632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-338f0e6f-c593-4476-8aec-c78251f24a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:37.764229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1055/2104] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:37 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:37.768663 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6ee2f-0172-4d54-8871-f5e0582c503b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.818498 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:37.873171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338f0e6f-c593-4476-8aec-c78251f24a10 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.902028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b6ee2f-0172-4d54-8871-f5e0582c503b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.907521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7df592-5d7f-46f2-855a-d964096ea250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.908264 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7df592-5d7f-46f2-855a-d964096ea250 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.910240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7df592-5d7f-46f2-855a-d964096ea250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.910536 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7df592-5d7f-46f2-855a-d964096ea250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.911294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7df592-5d7f-46f2-855a-d964096ea250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.913010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7df592-5d7f-46f2-855a-d964096ea250 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.925920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1056/2105] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:37.926787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7df592-5d7f-46f2-855a-d964096ea250 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.938682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7df592-5d7f-46f2-855a-d964096ea250 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.944830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fdfdd-d7ac-4d9c-b7c5-0b5834f9df7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.945588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fdfdd-d7ac-4d9c-b7c5-0b5834f9df7e admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.945859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fdfdd-d7ac-4d9c-b7c5-0b5834f9df7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.946154 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95fdfdd-d7ac-4d9c-b7c5-0b5834f9df7e admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.946864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fdfdd-d7ac-4d9c-b7c5-0b5834f9df7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.948831 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95fdfdd-d7ac-4d9c-b7c5-0b5834f9df7e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.954249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08b9a3-601d-4f82-9b26-82c1f8950cbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.954998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08b9a3-601d-4f82-9b26-82c1f8950cbe service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.955445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08b9a3-601d-4f82-9b26-82c1f8950cbe service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.955754 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a08b9a3-601d-4f82-9b26-82c1f8950cbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:37.968730 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08b9a3-601d-4f82-9b26-82c1f8950cbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:37.970975 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a08b9a3-601d-4f82-9b26-82c1f8950cbe service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:37.978030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1057/2106] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:37 2022] POST /identity/v3/users => generated 291 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:37.979809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fdfdd-d7ac-4d9c-b7c5-0b5834f9df7e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.988284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1050/2107] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:37.990535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08b9a3-601d-4f82-9b26-82c1f8950cbe service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:37.992628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95fdfdd-d7ac-4d9c-b7c5-0b5834f9df7e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:37.998320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dd6f9d-680d-4f11-bf37-baf629461150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:37.999278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd6f9d-680d-4f11-bf37-baf629461150 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:37.999563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd6f9d-680d-4f11-bf37-baf629461150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:37.999841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dd6f9d-680d-4f11-bf37-baf629461150 admin admin] PATH_INFO: `/v3/projects/80b39a505f1245f18c1ccd0b4117ce89/users/96deb8c34fee4c97ae12143fb64c510f/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.001420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a08b9a3-601d-4f82-9b26-82c1f8950cbe service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.009893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f371eff-3b3f-4b90-acaf-1cc9dc889927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.010652 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f371eff-3b3f-4b90-acaf-1cc9dc889927 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.010943 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f371eff-3b3f-4b90-acaf-1cc9dc889927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.011274 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f371eff-3b3f-4b90-acaf-1cc9dc889927 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.012518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f371eff-3b3f-4b90-acaf-1cc9dc889927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.014926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f371eff-3b3f-4b90-acaf-1cc9dc889927 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.021789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd6f9d-680d-4f11-bf37-baf629461150 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80b39a505f1245f18c1ccd0b4117ce89, user_id=96deb8c34fee4c97ae12143fb64c510f, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.025700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dd6f9d-680d-4f11-bf37-baf629461150 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.030375 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1051/2108] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:38.035130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f371eff-3b3f-4b90-acaf-1cc9dc889927 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.046476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f371eff-3b3f-4b90-acaf-1cc9dc889927 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.050695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1058/2109] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:37 2022] PUT /identity/v3/projects/80b39a505f1245f18c1ccd0b4117ce89/users/96deb8c34fee4c97ae12143fb64c510f/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:38.070058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb9e8-e803-4636-891c-32040076469e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.070757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb9e8-e803-4636-891c-32040076469e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.071203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fccb9e8-e803-4636-891c-32040076469e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.142076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0fccb9e8-e803-4636-891c-32040076469e None None] MFA Rules not processed for user `96deb8c34fee4c97ae12143fb64c510f`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:38.145655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0fccb9e8-e803-4636-891c-32040076469e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:38.183124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c818a1-abd2-4235-b148-0ff812f36c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.183861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c818a1-abd2-4235-b148-0ff812f36c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.184180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c818a1-abd2-4235-b148-0ff812f36c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.184460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c818a1-abd2-4235-b148-0ff812f36c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.185465 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c818a1-abd2-4235-b148-0ff812f36c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.187876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c818a1-abd2-4235-b148-0ff812f36c68 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.201736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1052/2110] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:28:38.206377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c818a1-abd2-4235-b148-0ff812f36c68 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.220834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c818a1-abd2-4235-b148-0ff812f36c68 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.228476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2e97a-d95a-4723-b39f-b64f13b41808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.229541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2e97a-d95a-4723-b39f-b64f13b41808 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.229938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2e97a-d95a-4723-b39f-b64f13b41808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.233734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2e97a-d95a-4723-b39f-b64f13b41808 admin admin] PATH_INFO: `/v3/projects/40d80cd516ba428f87fbf069b3e1548f/users/41b2f5377bf94604981077087af10e30/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.289648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2e97a-d95a-4723-b39f-b64f13b41808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40d80cd516ba428f87fbf069b3e1548f, user_id=41b2f5377bf94604981077087af10e30, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.290413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1059/2111] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:38.292882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2e97a-d95a-4723-b39f-b64f13b41808 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.298655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccb9e8-e803-4636-891c-32040076469e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.321616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fccb9e8-e803-4636-891c-32040076469e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.323759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1053/2112] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:38 2022] PUT /identity/v3/projects/40d80cd516ba428f87fbf069b3e1548f/users/41b2f5377bf94604981077087af10e30/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:38.337034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc948b95-c5d8-468c-8cb4-487d89862ddb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.337851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc948b95-c5d8-468c-8cb4-487d89862ddb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.338361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc948b95-c5d8-468c-8cb4-487d89862ddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.420880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:38.427581 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bc948b95-c5d8-468c-8cb4-487d89862ddb None None] MFA Rules not processed for user `41b2f5377bf94604981077087af10e30`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:38.430902 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bc948b95-c5d8-468c-8cb4-487d89862ddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:38.446578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:38.459418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95a77f-0797-4ac2-ab9f-33b1939d788c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.460301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95a77f-0797-4ac2-ab9f-33b1939d788c admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.460851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95a77f-0797-4ac2-ab9f-33b1939d788c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.461210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b95a77f-0797-4ac2-ab9f-33b1939d788c admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.462528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95a77f-0797-4ac2-ab9f-33b1939d788c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.464921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b95a77f-0797-4ac2-ab9f-33b1939d788c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.501575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1060/2113] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:38 2022] POST /identity/v3/projects => generated 383 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:38.506927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95a77f-0797-4ac2-ab9f-33b1939d788c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.584106 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1054/2114] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:38.587511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc948b95-c5d8-468c-8cb4-487d89862ddb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.612409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc948b95-c5d8-468c-8cb4-487d89862ddb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.640258 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b95a77f-0797-4ac2-ab9f-33b1939d788c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.662746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d68d38-9dc9-47a7-a3e1-5a479e0ec892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.663642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d68d38-9dc9-47a7-a3e1-5a479e0ec892 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.664130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d68d38-9dc9-47a7-a3e1-5a479e0ec892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.664566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d68d38-9dc9-47a7-a3e1-5a479e0ec892 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.665467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d68d38-9dc9-47a7-a3e1-5a479e0ec892 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.667649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d68d38-9dc9-47a7-a3e1-5a479e0ec892 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.730645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1061/2115] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:38 2022] POST /identity/v3/users => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:38.737649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d68d38-9dc9-47a7-a3e1-5a479e0ec892 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.777689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d68d38-9dc9-47a7-a3e1-5a479e0ec892 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.787473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9be1b9-4e2d-4cda-9e1c-1d5439721172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.789140 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9be1b9-4e2d-4cda-9e1c-1d5439721172 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.789766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9be1b9-4e2d-4cda-9e1c-1d5439721172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.790244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9be1b9-4e2d-4cda-9e1c-1d5439721172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.795281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9be1b9-4e2d-4cda-9e1c-1d5439721172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.811429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9be1b9-4e2d-4cda-9e1c-1d5439721172 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.816683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5f8d8-ebb0-4ea9-9f80-e178f013b42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.817411 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5f8d8-ebb0-4ea9-9f80-e178f013b42e admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.817674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5f8d8-ebb0-4ea9-9f80-e178f013b42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.825815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f5f8d8-ebb0-4ea9-9f80-e178f013b42e admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.826790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5f8d8-ebb0-4ea9-9f80-e178f013b42e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.830104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f5f8d8-ebb0-4ea9-9f80-e178f013b42e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.837758 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1062/2116] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:38 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:38.838641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9be1b9-4e2d-4cda-9e1c-1d5439721172 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.848291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5a9be1b9-4e2d-4cda-9e1c-1d5439721172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:38.885271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9be1b9-4e2d-4cda-9e1c-1d5439721172 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:38.893540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:38.894432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:38.894723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:38.895003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] PATH_INFO: `/v3/projects/429f4dad1ff3413bb1f117ae2c50b558` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:38.896579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429f4dad1ff3413bb1f117ae2c50b558)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:38.898775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:38.909188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1055/2117] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:38 2022] POST /identity/v3/projects => generated 383 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:28:38.910459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5f8d8-ebb0-4ea9-9f80-e178f013b42e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:38.979726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429f4dad1ff3413bb1f117ae2c50b558', 'request_id': 'req-138bfd96-4582-4495-a511-71e277759a71'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.004617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429f4dad1ff3413bb1f117ae2c50b558', 'request_id': 'req-138bfd96-4582-4495-a511-71e277759a71'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.020319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429f4dad1ff3413bb1f117ae2c50b558', 'request_id': 'req-138bfd96-4582-4495-a511-71e277759a71'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.036579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] The token cache is being invalidate because project 429f4dad1ff3413bb1f117ae2c50b558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:39.036887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.037721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.039629 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1063/2118] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:38 2022] DELETE /identity/v3/projects/429f4dad1ff3413bb1f117ae2c50b558 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:39.040484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.041943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:39.052347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f5f8d8-ebb0-4ea9-9f80-e178f013b42e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.132712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:39.136394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:39.179879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bfd96-4582-4495-a511-71e277759a71 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.187148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0aa4d0-6ee8-4da9-ae25-7159b30fc49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.189098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0aa4d0-6ee8-4da9-ae25-7159b30fc49c admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.189535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0aa4d0-6ee8-4da9-ae25-7159b30fc49c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.189823 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0aa4d0-6ee8-4da9-ae25-7159b30fc49c admin admin] PATH_INFO: `/v3/projects/251341d5d8e34cb1934b604972011b6e/users/d41011e36e0e420ca6b03edf26aa7ffa/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.191675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbd579-7ef2-4760-98eb-56b20546e142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.192370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbd579-7ef2-4760-98eb-56b20546e142 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.193701 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbd579-7ef2-4760-98eb-56b20546e142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.194002 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbd579-7ef2-4760-98eb-56b20546e142 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.194723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbd579-7ef2-4760-98eb-56b20546e142 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.196780 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbd579-7ef2-4760-98eb-56b20546e142 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:39.213525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0aa4d0-6ee8-4da9-ae25-7159b30fc49c admin admin] RBAC: Authorizing `identity:create_grant(project_id=251341d5d8e34cb1934b604972011b6e, user_id=d41011e36e0e420ca6b03edf26aa7ffa, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.216774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0aa4d0-6ee8-4da9-ae25-7159b30fc49c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:39.246264 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1056/2119] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:38 2022] PUT /identity/v3/projects/251341d5d8e34cb1934b604972011b6e/users/d41011e36e0e420ca6b03edf26aa7ffa/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:39.247434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0aa4d0-6ee8-4da9-ae25-7159b30fc49c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.250439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9d0aa4d0-6ee8-4da9-ae25-7159b30fc49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:39.261444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1064/2120] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:39 2022] POST /identity/v3/users => generated 298 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:39.261868 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbd579-7ef2-4760-98eb-56b20546e142 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.263966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4ddbd579-7ef2-4760-98eb-56b20546e142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:39.279754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbd579-7ef2-4760-98eb-56b20546e142 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.331688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0aa4d0-6ee8-4da9-ae25-7159b30fc49c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.337043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.337973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.338258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.338535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] PATH_INFO: `/v3/users/23a9fb25e87c453db58599824468e0b8` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.340050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a9fb25e87c453db58599824468e0b8)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.341987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:39.371414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb583db7-5e50-4245-9b36-e3e7d3842daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.371414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb583db7-5e50-4245-9b36-e3e7d3842daa admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.371414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb583db7-5e50-4245-9b36-e3e7d3842daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.371414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb583db7-5e50-4245-9b36-e3e7d3842daa admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.371864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb583db7-5e50-4245-9b36-e3e7d3842daa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.371864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb583db7-5e50-4245-9b36-e3e7d3842daa admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:39.390014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1065/2121] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:39.395155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb583db7-5e50-4245-9b36-e3e7d3842daa admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.412321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb583db7-5e50-4245-9b36-e3e7d3842daa admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.418436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461c593-a78a-439c-a2a6-7994e0ddf99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.421152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c593-a78a-439c-a2a6-7994e0ddf99a admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.421479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c593-a78a-439c-a2a6-7994e0ddf99a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.421767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6461c593-a78a-439c-a2a6-7994e0ddf99a admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.422547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461c593-a78a-439c-a2a6-7994e0ddf99a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.425127 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6461c593-a78a-439c-a2a6-7994e0ddf99a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:39.430935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a9fb25e87c453db58599824468e0b8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.444425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a9fb25e87c453db58599824468e0b8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.445526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a9fb25e87c453db58599824468e0b8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.451570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1066/2122] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:39.451570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461c593-a78a-439c-a2a6-7994e0ddf99a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.467085 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a9fb25e87c453db58599824468e0b8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.469642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6461c593-a78a-439c-a2a6-7994e0ddf99a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.479149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d78652-6915-4860-a31b-1356254ee118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.480773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78652-6915-4860-a31b-1356254ee118 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.481239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a9fb25e87c453db58599824468e0b8'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:39.481355 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78652-6915-4860-a31b-1356254ee118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.481835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d78652-6915-4860-a31b-1356254ee118 admin admin] PATH_INFO: `/v3/projects/251341d5d8e34cb1934b604972011b6e/users/d41011e36e0e420ca6b03edf26aa7ffa/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.487903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d78652-6915-4860-a31b-1356254ee118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=251341d5d8e34cb1934b604972011b6e, user_id=d41011e36e0e420ca6b03edf26aa7ffa, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.491756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d78652-6915-4860-a31b-1356254ee118 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:39.498811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1057/2123] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:39 2022] DELETE /identity/v3/users/23a9fb25e87c453db58599824468e0b8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:39.499570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.508824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:39.533409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1067/2124] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:39 2022] PUT /identity/v3/projects/251341d5d8e34cb1934b604972011b6e/users/d41011e36e0e420ca6b03edf26aa7ffa/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:39.544310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe68977-9c4b-4ae4-baf3-087b181b8513 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.544835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe68977-9c4b-4ae4-baf3-087b181b8513 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.545496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe68977-9c4b-4ae4-baf3-087b181b8513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.601672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:39.631433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ebe68977-9c4b-4ae4-baf3-087b181b8513 None None] MFA Rules not processed for user `d41011e36e0e420ca6b03edf26aa7ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:39.637672 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ebe68977-9c4b-4ae4-baf3-087b181b8513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:39.640694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf7e9e9-6f96-42e3-81a2-c74058995167 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.718538 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239bfd0-9fbc-4505-8a22-7fe05c0c41a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.719469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239bfd0-9fbc-4505-8a22-7fe05c0c41a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.719754 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239bfd0-9fbc-4505-8a22-7fe05c0c41a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.720033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0239bfd0-9fbc-4505-8a22-7fe05c0c41a2 admin admin] PATH_INFO: `/v3/projects/8ae628c4afa348f69f1361407757f547/users/9a6632e3526548ada358c7dd9b213fb8/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.740009 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239bfd0-9fbc-4505-8a22-7fe05c0c41a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae628c4afa348f69f1361407757f547, user_id=9a6632e3526548ada358c7dd9b213fb8, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.743110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0239bfd0-9fbc-4505-8a22-7fe05c0c41a2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:39.767149 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1058/2125] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:39 2022] PUT /identity/v3/projects/8ae628c4afa348f69f1361407757f547/users/9a6632e3526548ada358c7dd9b213fb8/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:39.773883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239bfd0-9fbc-4505-8a22-7fe05c0c41a2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.792148 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1068/2126] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:39.807838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe68977-9c4b-4ae4-baf3-087b181b8513 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.815191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ebe68977-9c4b-4ae4-baf3-087b181b8513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:39.899336 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0239bfd0-9fbc-4505-8a22-7fe05c0c41a2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.912512 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7795b-a48e-464e-8a32-0281a5883cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.913836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7795b-a48e-464e-8a32-0281a5883cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.915199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7795b-a48e-464e-8a32-0281a5883cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.916549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7795b-a48e-464e-8a32-0281a5883cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.917457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7795b-a48e-464e-8a32-0281a5883cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.925346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7795b-a48e-464e-8a32-0281a5883cf2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:39.950287 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1059/2127] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:28:39.951789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7795b-a48e-464e-8a32-0281a5883cf2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:39.972451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:39.974141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7795b-a48e-464e-8a32-0281a5883cf2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:39.987428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5ea7d-50a5-4bdd-a94a-2e4a2888a68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:39.989420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5ea7d-50a5-4bdd-a94a-2e4a2888a68b admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:39.989950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5ea7d-50a5-4bdd-a94a-2e4a2888a68b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:39.990315 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5ea7d-50a5-4bdd-a94a-2e4a2888a68b admin admin] PATH_INFO: `/v3/projects/8ae628c4afa348f69f1361407757f547/users/9a6632e3526548ada358c7dd9b213fb8/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:39.993217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5ea7d-50a5-4bdd-a94a-2e4a2888a68b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ae628c4afa348f69f1361407757f547, user_id=9a6632e3526548ada358c7dd9b213fb8, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:39.996536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5ea7d-50a5-4bdd-a94a-2e4a2888a68b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:40.014034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1060/2128] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:39 2022] PUT /identity/v3/projects/8ae628c4afa348f69f1361407757f547/users/9a6632e3526548ada358c7dd9b213fb8/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:40.030262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed9b08-89af-4a30-85ff-a20cdf749944 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.030765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed9b08-89af-4a30-85ff-a20cdf749944 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.031332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ed9b08-89af-4a30-85ff-a20cdf749944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.044725 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe68977-9c4b-4ae4-baf3-087b181b8513 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:40.108708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-58ed9b08-89af-4a30-85ff-a20cdf749944 None None] MFA Rules not processed for user `9a6632e3526548ada358c7dd9b213fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:40.110336 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-58ed9b08-89af-4a30-85ff-a20cdf749944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:40.128235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a6496-c02c-41b2-a81d-5e2d54f67c55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:40.129294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a6496-c02c-41b2-a81d-5e2d54f67c55 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.129928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a6496-c02c-41b2-a81d-5e2d54f67c55 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.130352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a6496-c02c-41b2-a81d-5e2d54f67c55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.133259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-571a6496-c02c-41b2-a81d-5e2d54f67c55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:40.186277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:40.277981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1061/2129] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:40.284468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a6496-c02c-41b2-a81d-5e2d54f67c55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:40.286347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571a6496-c02c-41b2-a81d-5e2d54f67c55 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:40.296286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed9b08-89af-4a30-85ff-a20cdf749944 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:40.298751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-58ed9b08-89af-4a30-85ff-a20cdf749944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:40.299687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1069/2130] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:28:40.316128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ed9b08-89af-4a30-85ff-a20cdf749944 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:40.321576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e7c64-fc57-4fdf-9a48-ec6ba0b08098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:40.322430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e7c64-fc57-4fdf-9a48-ec6ba0b08098 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.322853 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e7c64-fc57-4fdf-9a48-ec6ba0b08098 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.323390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e7c64-fc57-4fdf-9a48-ec6ba0b08098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.336196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e7c64-fc57-4fdf-9a48-ec6ba0b08098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:40.337944 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81e7c64-fc57-4fdf-9a48-ec6ba0b08098 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:40.355885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1062/2131] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:40.461021 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a6496-c02c-41b2-a81d-5e2d54f67c55 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:40.471307 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a6496-c02c-41b2-a81d-5e2d54f67c55 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:40.475930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:40.476611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:40.476904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:40.477190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:40.479363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:40.586020 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:40.588938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:40.601017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1070/2132] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:41.102341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e7c64-fc57-4fdf-9a48-ec6ba0b08098 service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.188041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.191607 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:41.263260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81e7c64-fc57-4fdf-9a48-ec6ba0b08098 service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.279008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.279993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.280279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.280568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] PATH_INFO: `/v3/users/41b2f5377bf94604981077087af10e30` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.281917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b2f5377bf94604981077087af10e30)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:41.284276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:41.292382 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:41.348518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85081b59-4f5e-48b0-a630-e1c44e90ad04 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.355255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.356234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.356681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.357047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] PATH_INFO: `/v3/projects/169a45fb4c7b41ee8b28f84ede0a33a1` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.358376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=169a45fb4c7b41ee8b28f84ede0a33a1)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:41.360746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:41.378856 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b2f5377bf94604981077087af10e30'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.394799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b2f5377bf94604981077087af10e30'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.397585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b2f5377bf94604981077087af10e30'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.433638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b2f5377bf94604981077087af10e30'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.458010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-44f9406d-cb03-40ce-8f71-a3f8f2fd1f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b2f5377bf94604981077087af10e30'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.467073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '169a45fb4c7b41ee8b28f84ede0a33a1', 'request_id': 'req-3ce17a12-36f1-41aa-857c-75fd7c2d6359'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.477472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1063/2133] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:41 2022] DELETE /identity/v3/users/41b2f5377bf94604981077087af10e30 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:41.477766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '169a45fb4c7b41ee8b28f84ede0a33a1', 'request_id': 'req-3ce17a12-36f1-41aa-857c-75fd7c2d6359'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.483693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b9cb4-823f-468a-8b23-0c6f57827673 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.484160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b9cb4-823f-468a-8b23-0c6f57827673 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.484558 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b9cb4-823f-468a-8b23-0c6f57827673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.490595 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '169a45fb4c7b41ee8b28f84ede0a33a1', 'request_id': 'req-3ce17a12-36f1-41aa-857c-75fd7c2d6359'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.511916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] The token cache is being invalidate because project 169a45fb4c7b41ee8b28f84ede0a33a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:41.512514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.513881 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.517505 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1071/2134] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:41 2022] DELETE /identity/v3/projects/169a45fb4c7b41ee8b28f84ede0a33a1 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:41.520441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-142b9cb4-823f-468a-8b23-0c6f57827673 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:41.522214 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-142b9cb4-823f-468a-8b23-0c6f57827673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:41.522425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.523830 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:41.641822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:41.693091 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1064/2135] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:41.698864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce17a12-36f1-41aa-857c-75fd7c2d6359 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.702353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b9cb4-823f-468a-8b23-0c6f57827673 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:41.704046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-142b9cb4-823f-468a-8b23-0c6f57827673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:41.704351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.705467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.705760 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.706041 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] PATH_INFO: `/v3/users/cdeed567c36f4077ad2bc531afa80897` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.707762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdeed567c36f4077ad2bc531afa80897)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:41.709820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:41.781449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdeed567c36f4077ad2bc531afa80897'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.791511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdeed567c36f4077ad2bc531afa80897'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.793155 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdeed567c36f4077ad2bc531afa80897'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.816145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdeed567c36f4077ad2bc531afa80897'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.832744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:41.850493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b9cb4-823f-468a-8b23-0c6f57827673 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:41.851085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdeed567c36f4077ad2bc531afa80897'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:41.858194 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:41.859265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:41.859764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:41.860721 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:41.866706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:41.875644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1072/2136] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:41 2022] DELETE /identity/v3/users/cdeed567c36f4077ad2bc531afa80897 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:41.917174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:41.922930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:41.989488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1065/2137] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:42.226550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.229682 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:42.322855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838b2477-6ee2-4a74-9658-ec90311da954 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.327414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.328333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.328763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.329178 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.333171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:42.400754 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.404346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:42.438936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435bafb-b55f-4f73-828e-8b94421525a5 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.549323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.553224 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:42.596514 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.598545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.598862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.599151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] PATH_INFO: `/v3/users/e010b6d917ca464dbba63d822cfcff1d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.600331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1073/2138] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:42.602225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e010b6d917ca464dbba63d822cfcff1d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.604390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:42.685739 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e010b6d917ca464dbba63d822cfcff1d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:42.700775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e010b6d917ca464dbba63d822cfcff1d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:42.701764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e010b6d917ca464dbba63d822cfcff1d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:42.733120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e010b6d917ca464dbba63d822cfcff1d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:42.756888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-82d93841-0ccd-41b4-84d3-080b6bcec550 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e010b6d917ca464dbba63d822cfcff1d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:42.789325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:42.790897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1066/2139] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:42 2022] DELETE /identity/v3/users/e010b6d917ca464dbba63d822cfcff1d => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:42.791370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:42.796615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51933c32-b142-4f83-92dd-2e61e712b496 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.797867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51933c32-b142-4f83-92dd-2e61e712b496 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.799345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51933c32-b142-4f83-92dd-2e61e712b496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.825551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b444ada-2cf5-44ff-9c11-67672c13f0f4 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:42.831087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:42.860403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-51933c32-b142-4f83-92dd-2e61e712b496 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:42.862865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-51933c32-b142-4f83-92dd-2e61e712b496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:42.987960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:42.988975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:42.990389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:42.990389 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] PATH_INFO: `/v3/projects/40d80cd516ba428f87fbf069b3e1548f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:42.994968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40d80cd516ba428f87fbf069b3e1548f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:42.999648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.130759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1067/2140] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:43.133927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51933c32-b142-4f83-92dd-2e61e712b496 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.136910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-51933c32-b142-4f83-92dd-2e61e712b496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:43.156115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:43.161686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40d80cd516ba428f87fbf069b3e1548f', 'request_id': 'req-5f10290a-d180-45a2-85a2-cb2a4877d861'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.176763 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40d80cd516ba428f87fbf069b3e1548f', 'request_id': 'req-5f10290a-d180-45a2-85a2-cb2a4877d861'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.199495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40d80cd516ba428f87fbf069b3e1548f', 'request_id': 'req-5f10290a-d180-45a2-85a2-cb2a4877d861'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.240630 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] The token cache is being invalidate because project 40d80cd516ba428f87fbf069b3e1548f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:43.241297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.241848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.247259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1074/2141] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:42 2022] DELETE /identity/v3/projects/40d80cd516ba428f87fbf069b3e1548f => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:43.249544 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.251070 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:43.300842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51933c32-b142-4f83-92dd-2e61e712b496 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.417103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10290a-d180-45a2-85a2-cb2a4877d861 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.436621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.439066 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.439818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.440442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] PATH_INFO: `/v3/users/9a6632e3526548ada358c7dd9b213fb8` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.443032 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a6632e3526548ada358c7dd9b213fb8)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.451245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.454129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.456577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.457090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.457600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.463557 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:43.519212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.521727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.567556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1068/2142] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:43.569084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.571765 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:43.586644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a6632e3526548ada358c7dd9b213fb8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.603888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25cc9f-bc14-46ce-9932-e45a76e42d6b service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:43.612804 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:43.614325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a6632e3526548ada358c7dd9b213fb8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.615266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a6632e3526548ada358c7dd9b213fb8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.615427 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.615764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.616146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] PATH_INFO: `/v3/projects/7d6ef193a7c641d19b4425e91f781c00` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.617788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d6ef193a7c641d19b4425e91f781c00)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:43.620918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:43.630561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a6632e3526548ada358c7dd9b213fb8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.658336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3dd23235-6082-438d-b7ef-67444b808879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a6632e3526548ada358c7dd9b213fb8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.673641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1075/2143] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:43 2022] DELETE /identity/v3/users/9a6632e3526548ada358c7dd9b213fb8 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:43.730561 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d6ef193a7c641d19b4425e91f781c00', 'request_id': 'req-f14d7d31-2a82-4f62-9190-db0f62d05372'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.779316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d6ef193a7c641d19b4425e91f781c00', 'request_id': 'req-f14d7d31-2a82-4f62-9190-db0f62d05372'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.781801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:43.795999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d6ef193a7c641d19b4425e91f781c00', 'request_id': 'req-f14d7d31-2a82-4f62-9190-db0f62d05372'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.816179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] The token cache is being invalidate because project 7d6ef193a7c641d19b4425e91f781c00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:43.816468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.817356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:43.823347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1069/2144] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:43 2022] DELETE /identity/v3/projects/7d6ef193a7c641d19b4425e91f781c00 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:43.831042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46610a98-b32c-4abc-b27b-008ce1569f5c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:43.831836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46610a98-b32c-4abc-b27b-008ce1569f5c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:43.832381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46610a98-b32c-4abc-b27b-008ce1569f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:43.881225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:43.882940 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:43.895717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-46610a98-b32c-4abc-b27b-008ce1569f5c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:43.898690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-46610a98-b32c-4abc-b27b-008ce1569f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:44.039643 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d7d31-2a82-4f62-9190-db0f62d05372 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.046383 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59cc02-2414-46da-aa40-42c21a99eabc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.047360 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59cc02-2414-46da-aa40-42c21a99eabc service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.047761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59cc02-2414-46da-aa40-42c21a99eabc service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.048265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59cc02-2414-46da-aa40-42c21a99eabc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.053674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-aa59cc02-2414-46da-aa40-42c21a99eabc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:44.055722 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:44.098801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1076/2145] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:44.108144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6eccd-5aff-469d-9ca5-128d1bfa03ea None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.108582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6eccd-5aff-469d-9ca5-128d1bfa03ea None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.108863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6eccd-5aff-469d-9ca5-128d1bfa03ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.154534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-05b6eccd-5aff-469d-9ca5-128d1bfa03ea None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:44.155578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-05b6eccd-5aff-469d-9ca5-128d1bfa03ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:44.182771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1077/2146] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:44.188901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6eccd-5aff-469d-9ca5-128d1bfa03ea None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.223434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6eccd-5aff-469d-9ca5-128d1bfa03ea None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.240565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425e95e-257b-4591-af34-d618337d00b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.242236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425e95e-257b-4591-af34-d618337d00b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.242236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425e95e-257b-4591-af34-d618337d00b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.243162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f425e95e-257b-4591-af34-d618337d00b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.256152 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425e95e-257b-4591-af34-d618337d00b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:44.256582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425e95e-257b-4591-af34-d618337d00b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.262398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59cc02-2414-46da-aa40-42c21a99eabc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.264392 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59cc02-2414-46da-aa40-42c21a99eabc service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.265517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f425e95e-257b-4591-af34-d618337d00b7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.287112 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1070/2147] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:28:44.324381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1078/2148] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:44.326268 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59cc02-2414-46da-aa40-42c21a99eabc service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.349568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59cc02-2414-46da-aa40-42c21a99eabc service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.381861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e568c84-091c-47a8-9046-5a4baf24f55d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.383061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e568c84-091c-47a8-9046-5a4baf24f55d admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.383567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e568c84-091c-47a8-9046-5a4baf24f55d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.384003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e568c84-091c-47a8-9046-5a4baf24f55d admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.384871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e568c84-091c-47a8-9046-5a4baf24f55d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.387207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e568c84-091c-47a8-9046-5a4baf24f55d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.458973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1071/2149] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:44 2022] POST /identity/v3/projects => generated 377 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:44.463837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425e95e-257b-4591-af34-d618337d00b7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.488804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f425e95e-257b-4591-af34-d618337d00b7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.683190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e568c84-091c-47a8-9046-5a4baf24f55d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.685078 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5e568c84-091c-47a8-9046-5a4baf24f55d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:44.686531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad049d-9f7b-453e-b6eb-258f171ed94a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.687283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad049d-9f7b-453e-b6eb-258f171ed94a admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.687834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad049d-9f7b-453e-b6eb-258f171ed94a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.688538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad049d-9f7b-453e-b6eb-258f171ed94a admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.690531 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad049d-9f7b-453e-b6eb-258f171ed94a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.698008 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfad049d-9f7b-453e-b6eb-258f171ed94a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.706991 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e568c84-091c-47a8-9046-5a4baf24f55d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.739267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.740065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.740441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.741025 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] PATH_INFO: `/v3/projects/c1814ed244354c17b1ac7b1089a74bb7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.742749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1814ed244354c17b1ac7b1089a74bb7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.743741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1079/2150] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:44 2022] POST /identity/v3/users => generated 289 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:44.748683 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad049d-9f7b-453e-b6eb-258f171ed94a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.748993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.775913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad049d-9f7b-453e-b6eb-258f171ed94a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.793865 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30532ac-beae-4577-a6ff-53fa08207b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:44.796519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30532ac-beae-4577-a6ff-53fa08207b0e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:44.797305 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30532ac-beae-4577-a6ff-53fa08207b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:44.797973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30532ac-beae-4577-a6ff-53fa08207b0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:44.801698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30532ac-beae-4577-a6ff-53fa08207b0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:44.804289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30532ac-beae-4577-a6ff-53fa08207b0e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:44.830226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1080/2151] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:44.837219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1814ed244354c17b1ac7b1089a74bb7', 'request_id': 'req-6bc46401-5d39-4bcf-b119-c698e7092f7d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:44.843950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30532ac-beae-4577-a6ff-53fa08207b0e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.849176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1814ed244354c17b1ac7b1089a74bb7', 'request_id': 'req-6bc46401-5d39-4bcf-b119-c698e7092f7d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:44.862223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30532ac-beae-4577-a6ff-53fa08207b0e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:44.865436 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1814ed244354c17b1ac7b1089a74bb7', 'request_id': 'req-6bc46401-5d39-4bcf-b119-c698e7092f7d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:44.883017 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:44.896915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] The token cache is being invalidate because project c1814ed244354c17b1ac7b1089a74bb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:44.897346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:44.900297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:44.905015 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1072/2152] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:44 2022] DELETE /identity/v3/projects/c1814ed244354c17b1ac7b1089a74bb7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:44.907445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:44.916439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:45.014263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964ce4f-4513-4ce2-ab56-01f114dc0fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.015068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964ce4f-4513-4ce2-ab56-01f114dc0fdb admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.015358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964ce4f-4513-4ce2-ab56-01f114dc0fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.015633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964ce4f-4513-4ce2-ab56-01f114dc0fdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.016482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964ce4f-4513-4ce2-ab56-01f114dc0fdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.021443 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5964ce4f-4513-4ce2-ab56-01f114dc0fdb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:45.053662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1081/2153] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:28:45.058637 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964ce4f-4513-4ce2-ab56-01f114dc0fdb admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:45.060642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5964ce4f-4513-4ce2-ab56-01f114dc0fdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:45.127378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964ce4f-4513-4ce2-ab56-01f114dc0fdb admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.131909 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc46401-5d39-4bcf-b119-c698e7092f7d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.143473 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.143849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4676f-92b2-4277-ac4e-c5734ca19423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.144451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.144732 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.145005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] PATH_INFO: `/v3/users/5d40b53d8b37452797f1539b4acadbf4` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.145945 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4676f-92b2-4277-ac4e-c5734ca19423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.146715 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4676f-92b2-4277-ac4e-c5734ca19423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.147215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4676f-92b2-4277-ac4e-c5734ca19423 admin admin] PATH_INFO: `/v3/projects/1343502925e4438c87f823fc530a5a58/users/fdb3c23f23e84f4cbf78871e8c192ec7/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.148893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d40b53d8b37452797f1539b4acadbf4)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.151084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:45.208715 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4676f-92b2-4277-ac4e-c5734ca19423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1343502925e4438c87f823fc530a5a58, user_id=fdb3c23f23e84f4cbf78871e8c192ec7, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.211875 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4676f-92b2-4277-ac4e-c5734ca19423 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:45.252961 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d40b53d8b37452797f1539b4acadbf4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:45.271431 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1082/2154] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:45 2022] PUT /identity/v3/projects/1343502925e4438c87f823fc530a5a58/users/fdb3c23f23e84f4cbf78871e8c192ec7/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:45.271971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d40b53d8b37452797f1539b4acadbf4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:45.271971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d40b53d8b37452797f1539b4acadbf4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:45.274885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27b5b1-bb0f-454c-866f-6b2cb75cdb4c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.275318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27b5b1-bb0f-454c-866f-6b2cb75cdb4c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.275678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b27b5b1-bb0f-454c-866f-6b2cb75cdb4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.280406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d40b53d8b37452797f1539b4acadbf4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:45.289363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d40b53d8b37452797f1539b4acadbf4'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:45.302151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1073/2155] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:44 2022] DELETE /identity/v3/users/5d40b53d8b37452797f1539b4acadbf4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:45.346713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-9b27b5b1-bb0f-454c-866f-6b2cb75cdb4c None None] MFA Rules not processed for user `fdb3c23f23e84f4cbf78871e8c192ec7`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:45.347756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9b27b5b1-bb0f-454c-866f-6b2cb75cdb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:45.370327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:45.493921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1083/2156] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:45.507381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:45.509162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:45.646520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27b5b1-bb0f-454c-866f-6b2cb75cdb4c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:45.647964 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9b27b5b1-bb0f-454c-866f-6b2cb75cdb4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:45.649335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d2123-e719-432d-b4d2-1fe351190b28 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.657298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef891d5c-b546-42eb-93f3-d015f2569f5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.658348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef891d5c-b546-42eb-93f3-d015f2569f5b service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.659217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef891d5c-b546-42eb-93f3-d015f2569f5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.659548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef891d5c-b546-42eb-93f3-d015f2569f5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.684625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef891d5c-b546-42eb-93f3-d015f2569f5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.688482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef891d5c-b546-42eb-93f3-d015f2569f5b service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:45.725307 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1074/2157] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:45.823187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b27b5b1-bb0f-454c-866f-6b2cb75cdb4c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:45.833868 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:45.835068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:45.836568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:45.836983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] PATH_INFO: `/v3/projects/8ae628c4afa348f69f1361407757f547` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:45.840433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ae628c4afa348f69f1361407757f547)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:45.844405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.041037 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ae628c4afa348f69f1361407757f547', 'request_id': 'req-16a6e390-af78-42fc-8345-1268109168a8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.072911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ae628c4afa348f69f1361407757f547', 'request_id': 'req-16a6e390-af78-42fc-8345-1268109168a8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.098301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ae628c4afa348f69f1361407757f547', 'request_id': 'req-16a6e390-af78-42fc-8345-1268109168a8'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.132513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] The token cache is being invalidate because project 8ae628c4afa348f69f1361407757f547 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:46.132816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.136356 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16a6e390-af78-42fc-8345-1268109168a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.139227 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1084/2158] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:45 2022] DELETE /identity/v3/projects/8ae628c4afa348f69f1361407757f547 => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:46.148453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74a2e6-f57f-4b2f-afa5-568a3a2523cc None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.148770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74a2e6-f57f-4b2f-afa5-568a3a2523cc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.149129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c74a2e6-f57f-4b2f-afa5-568a3a2523cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.181108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1c74a2e6-f57f-4b2f-afa5-568a3a2523cc None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:46.184229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1c74a2e6-f57f-4b2f-afa5-568a3a2523cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:46.212135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:46.331682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1075/2159] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:46.347587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9c5b0-0f13-48d9-bb78-d1579713709a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.347587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9c5b0-0f13-48d9-bb78-d1579713709a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.347587 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9c5b0-0f13-48d9-bb78-d1579713709a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.351758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74a2e6-f57f-4b2f-afa5-568a3a2523cc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.354894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1c74a2e6-f57f-4b2f-afa5-568a3a2523cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:46.393977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c74a2e6-f57f-4b2f-afa5-568a3a2523cc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.408394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3de9c5b0-0f13-48d9-bb78-d1579713709a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:46.413935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3de9c5b0-0f13-48d9-bb78-d1579713709a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:46.417351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.431854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.433316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.433662 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] PATH_INFO: `/v3/projects/441ffbe20a794264a493c820b420c250` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.436796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=441ffbe20a794264a493c820b420c250)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:46.439138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.451413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1085/2160] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:46.456304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9c5b0-0f13-48d9-bb78-d1579713709a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.473099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9c5b0-0f13-48d9-bb78-d1579713709a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.480508 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9e279-27ce-4f20-aa27-c2bc84cc5f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.481488 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9e279-27ce-4f20-aa27-c2bc84cc5f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.481783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9e279-27ce-4f20-aa27-c2bc84cc5f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.482118 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f9e279-27ce-4f20-aa27-c2bc84cc5f65 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.482921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9e279-27ce-4f20-aa27-c2bc84cc5f65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:46.483185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9e279-27ce-4f20-aa27-c2bc84cc5f65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:46.484964 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f9e279-27ce-4f20-aa27-c2bc84cc5f65 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.516524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1086/2161] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:28:46.518253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9e279-27ce-4f20-aa27-c2bc84cc5f65 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.526212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '441ffbe20a794264a493c820b420c250', 'request_id': 'req-4c2960ae-2e44-466b-ab60-6b1446e937cf'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.532730 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f9e279-27ce-4f20-aa27-c2bc84cc5f65 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.533566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '441ffbe20a794264a493c820b420c250', 'request_id': 'req-4c2960ae-2e44-466b-ab60-6b1446e937cf'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.545596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '441ffbe20a794264a493c820b420c250', 'request_id': 'req-4c2960ae-2e44-466b-ab60-6b1446e937cf'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.564990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] The token cache is being invalidate because project 441ffbe20a794264a493c820b420c250 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:46.565417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.566057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:46.568932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1076/2162] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:46 2022] DELETE /identity/v3/projects/441ffbe20a794264a493c820b420c250 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:46.586265 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:46.650074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d268b-b41d-4f5e-9b0d-42683624fc5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.651146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d268b-b41d-4f5e-9b0d-42683624fc5c admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.651834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d268b-b41d-4f5e-9b0d-42683624fc5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.652473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d268b-b41d-4f5e-9b0d-42683624fc5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.653525 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d268b-b41d-4f5e-9b0d-42683624fc5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:46.655914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d268b-b41d-4f5e-9b0d-42683624fc5c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.694241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1087/2163] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:46 2022] POST /identity/v3/projects => generated 373 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:46.698365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.699924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:46.720022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2960ae-2e44-466b-ab60-6b1446e937cf admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.841366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bab4c-86a8-4597-8b6c-1d346efa6dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.841795 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bab4c-86a8-4597-8b6c-1d346efa6dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.842158 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bab4c-86a8-4597-8b6c-1d346efa6dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.842554 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5bab4c-86a8-4597-8b6c-1d346efa6dd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.843352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5bab4c-86a8-4597-8b6c-1d346efa6dd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:46.845409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5bab4c-86a8-4597-8b6c-1d346efa6dd0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.869834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1077/2164] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:46 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:46.873754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d268b-b41d-4f5e-9b0d-42683624fc5c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.888143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d268b-b41d-4f5e-9b0d-42683624fc5c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.894263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3369-bd9b-4ab5-8b4a-efecf00df7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.895579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3369-bd9b-4ab5-8b4a-efecf00df7d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.895697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3369-bd9b-4ab5-8b4a-efecf00df7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.896387 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba3369-bd9b-4ab5-8b4a-efecf00df7d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.897936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba3369-bd9b-4ab5-8b4a-efecf00df7d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:46.900548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba3369-bd9b-4ab5-8b4a-efecf00df7d9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.914241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1088/2165] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:46.918134 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bab4c-86a8-4597-8b6c-1d346efa6dd0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.938920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5bab4c-86a8-4597-8b6c-1d346efa6dd0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.948236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb39bf6-251f-4a3b-9466-7239ec6b3ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.954679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb39bf6-251f-4a3b-9466-7239ec6b3ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.955531 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb39bf6-251f-4a3b-9466-7239ec6b3ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.955826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb39bf6-251f-4a3b-9466-7239ec6b3ba1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:46.957527 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb39bf6-251f-4a3b-9466-7239ec6b3ba1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:46.958866 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb39bf6-251f-4a3b-9466-7239ec6b3ba1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:46.971323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1078/2166] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:46.976309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3369-bd9b-4ab5-8b4a-efecf00df7d9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:46.987575 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba3369-bd9b-4ab5-8b4a-efecf00df7d9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:46.992441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b684d-4b23-4849-9130-07572ec6b77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:46.993343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b684d-4b23-4849-9130-07572ec6b77e admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:46.993654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b684d-4b23-4849-9130-07572ec6b77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:46.993934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629b684d-4b23-4849-9130-07572ec6b77e admin admin] PATH_INFO: `/v3/projects/a7bf50e110e740a2900febabade6f840/users/0013c27e2a2b440cb6a57bad2a338398/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.011601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b684d-4b23-4849-9130-07572ec6b77e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7bf50e110e740a2900febabade6f840, user_id=0013c27e2a2b440cb6a57bad2a338398, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:47.014581 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629b684d-4b23-4849-9130-07572ec6b77e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:47.031879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1089/2167] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:46 2022] PUT /identity/v3/projects/a7bf50e110e740a2900febabade6f840/users/0013c27e2a2b440cb6a57bad2a338398/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:47.038386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a9351-01f3-4394-8284-dce9ece0275f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:47.039075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a9351-01f3-4394-8284-dce9ece0275f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:47.039075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a9351-01f3-4394-8284-dce9ece0275f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.060117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b05a9351-01f3-4394-8284-dce9ece0275f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:47.061418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b05a9351-01f3-4394-8284-dce9ece0275f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:47.094111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:47.179423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1079/2168] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:47.192887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b684d-4b23-4849-9130-07572ec6b77e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:47.194748 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-629b684d-4b23-4849-9130-07572ec6b77e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:47.286045 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629b684d-4b23-4849-9130-07572ec6b77e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:47.290562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ee084-6b09-4a40-91b7-c73ddb839890 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:47.291230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ee084-6b09-4a40-91b7-c73ddb839890 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:47.292237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ee084-6b09-4a40-91b7-c73ddb839890 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:47.292570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ee084-6b09-4a40-91b7-c73ddb839890 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:47.306497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ee084-6b09-4a40-91b7-c73ddb839890 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:47.308042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ee084-6b09-4a40-91b7-c73ddb839890 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:47.321959 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1090/2169] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:48.175639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636ab3c-981c-4e79-9d4e-078c2b2391bd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.176221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636ab3c-981c-4e79-9d4e-078c2b2391bd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.176221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636ab3c-981c-4e79-9d4e-078c2b2391bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.215345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-9636ab3c-981c-4e79-9d4e-078c2b2391bd None None] MFA Rules not processed for user `0013c27e2a2b440cb6a57bad2a338398`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:48.216596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9636ab3c-981c-4e79-9d4e-078c2b2391bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:48.305061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1080/2170] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:48.312011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ee084-6b09-4a40-91b7-c73ddb839890 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.326367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ee084-6b09-4a40-91b7-c73ddb839890 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.333330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9e3b1-078c-45bd-a78a-b697c1a985e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.334036 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9e3b1-078c-45bd-a78a-b697c1a985e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.334326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9e3b1-078c-45bd-a78a-b697c1a985e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.334612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d9e3b1-078c-45bd-a78a-b697c1a985e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.335325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9e3b1-078c-45bd-a78a-b697c1a985e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:48.337522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d9e3b1-078c-45bd-a78a-b697c1a985e3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:48.373422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1091/2171] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:48 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:48.375716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636ab3c-981c-4e79-9d4e-078c2b2391bd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.387356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636ab3c-981c-4e79-9d4e-078c2b2391bd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.453410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee766d-a2fa-48d8-8180-e81abff53b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.454093 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee766d-a2fa-48d8-8180-e81abff53b48 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.454376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee766d-a2fa-48d8-8180-e81abff53b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.454654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ee766d-a2fa-48d8-8180-e81abff53b48 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.455909 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee766d-a2fa-48d8-8180-e81abff53b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:48.457999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ee766d-a2fa-48d8-8180-e81abff53b48 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:48.483304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1081/2172] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:48 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:48.484626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9e3b1-078c-45bd-a78a-b697c1a985e3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.495566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d9e3b1-078c-45bd-a78a-b697c1a985e3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.502595 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18213-380f-417b-a39b-e7130898f2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.503303 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18213-380f-417b-a39b-e7130898f2d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.503665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18213-380f-417b-a39b-e7130898f2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.503946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18213-380f-417b-a39b-e7130898f2d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.504692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c18213-380f-417b-a39b-e7130898f2d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:48.506455 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c18213-380f-417b-a39b-e7130898f2d4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:48.519071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1092/2173] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:48.525651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee766d-a2fa-48d8-8180-e81abff53b48 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.539876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ee766d-a2fa-48d8-8180-e81abff53b48 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.545338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c991c9-0d90-4554-b759-ad121da1698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.546779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c991c9-0d90-4554-b759-ad121da1698c admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.547141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c991c9-0d90-4554-b759-ad121da1698c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.547488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c991c9-0d90-4554-b759-ad121da1698c admin admin] PATH_INFO: `/v3/projects/db6f65dcd8cf4e179457181d75ff2f75/users/b747ba8ddead4b788b8c4a33b2df96b5/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.572256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c991c9-0d90-4554-b759-ad121da1698c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db6f65dcd8cf4e179457181d75ff2f75, user_id=b747ba8ddead4b788b8c4a33b2df96b5, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:48.587068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c991c9-0d90-4554-b759-ad121da1698c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:48.604808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1082/2174] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:48 2022] PUT /identity/v3/projects/db6f65dcd8cf4e179457181d75ff2f75/users/b747ba8ddead4b788b8c4a33b2df96b5/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:48.609126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18213-380f-417b-a39b-e7130898f2d4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.620759 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18213-380f-417b-a39b-e7130898f2d4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.677417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:48.709623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f171c86-9e35-47a0-83db-f731b1e68ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.710438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f171c86-9e35-47a0-83db-f731b1e68ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.710874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f171c86-9e35-47a0-83db-f731b1e68ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.711341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f171c86-9e35-47a0-83db-f731b1e68ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.713052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f171c86-9e35-47a0-83db-f731b1e68ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:48.716877 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f171c86-9e35-47a0-83db-f731b1e68ccf admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:48.730111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1093/2175] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:28:48.735271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c991c9-0d90-4554-b759-ad121da1698c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.746655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c991c9-0d90-4554-b759-ad121da1698c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:48.753829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8357a779-3619-4cd7-9d4f-97fb9acd06a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:48.754871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357a779-3619-4cd7-9d4f-97fb9acd06a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:48.755243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357a779-3619-4cd7-9d4f-97fb9acd06a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:48.755529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8357a779-3619-4cd7-9d4f-97fb9acd06a7 admin admin] PATH_INFO: `/v3/projects/db6f65dcd8cf4e179457181d75ff2f75/users/b747ba8ddead4b788b8c4a33b2df96b5/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:48.757636 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357a779-3619-4cd7-9d4f-97fb9acd06a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db6f65dcd8cf4e179457181d75ff2f75, user_id=b747ba8ddead4b788b8c4a33b2df96b5, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:48.760834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8357a779-3619-4cd7-9d4f-97fb9acd06a7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:48.776292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1083/2176] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:48 2022] PUT /identity/v3/projects/db6f65dcd8cf4e179457181d75ff2f75/users/b747ba8ddead4b788b8c4a33b2df96b5/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:48.939245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f171c86-9e35-47a0-83db-f731b1e68ccf admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:48.941206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6f171c86-9e35-47a0-83db-f731b1e68ccf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:48.963865 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f171c86-9e35-47a0-83db-f731b1e68ccf admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.062428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.063609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.063890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.064764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] PATH_INFO: `/v3/users/a409bd6c4dc24a15ace28d58282c41bb` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.066265 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a409bd6c4dc24a15ace28d58282c41bb)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:49.068271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:49.130138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a409bd6c4dc24a15ace28d58282c41bb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.138237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a409bd6c4dc24a15ace28d58282c41bb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.139018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a409bd6c4dc24a15ace28d58282c41bb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.150900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a409bd6c4dc24a15ace28d58282c41bb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.165315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a409bd6c4dc24a15ace28d58282c41bb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.172141 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1094/2177] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:48 2022] DELETE /identity/v3/users/a409bd6c4dc24a15ace28d58282c41bb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:49.178376 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b59293-dc89-4100-98b8-bc1f94eb82ee None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.178896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b59293-dc89-4100-98b8-bc1f94eb82ee None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.179154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b59293-dc89-4100-98b8-bc1f94eb82ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.186286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:49.199548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-85b59293-dc89-4100-98b8-bc1f94eb82ee None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:49.201846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-85b59293-dc89-4100-98b8-bc1f94eb82ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:49.315789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1084/2178] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:49.329745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.332281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:49.426667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf86fc-1849-4f9c-bbce-e4abfd8be948 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.439364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ef71e-d7be-4fed-b030-412c18c6956a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.440106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ef71e-d7be-4fed-b030-412c18c6956a service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.440436 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ef71e-d7be-4fed-b030-412c18c6956a service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.441179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ef71e-d7be-4fed-b030-412c18c6956a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.453626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ef71e-d7be-4fed-b030-412c18c6956a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:49.455718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ef71e-d7be-4fed-b030-412c18c6956a service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:49.468337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1095/2179] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:49.798346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b59293-dc89-4100-98b8-bc1f94eb82ee None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.800931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-85b59293-dc89-4100-98b8-bc1f94eb82ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:49.818168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b59293-dc89-4100-98b8-bc1f94eb82ee None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:49.825440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:49.826695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.826926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.827290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] PATH_INFO: `/v3/projects/8d10f59a27974c319846702d6b9a6944` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.830920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d10f59a27974c319846702d6b9a6944)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:49.831536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:49.879981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471d8b0-448a-44d7-a10c-a2802d9082a0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:49.880276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471d8b0-448a-44d7-a10c-a2802d9082a0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:49.880577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3471d8b0-448a-44d7-a10c-a2802d9082a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:49.887597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:49.922049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3471d8b0-448a-44d7-a10c-a2802d9082a0 None None] MFA Rules not processed for user `b747ba8ddead4b788b8c4a33b2df96b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:49.923468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3471d8b0-448a-44d7-a10c-a2802d9082a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:49.932868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d10f59a27974c319846702d6b9a6944', 'request_id': 'req-d40d01b2-52c1-4e02-94fd-426ff43788b2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.945104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d10f59a27974c319846702d6b9a6944', 'request_id': 'req-d40d01b2-52c1-4e02-94fd-426ff43788b2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.957107 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d10f59a27974c319846702d6b9a6944', 'request_id': 'req-d40d01b2-52c1-4e02-94fd-426ff43788b2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.970322 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] The token cache is being invalidate because project 8d10f59a27974c319846702d6b9a6944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:49.971910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.972246 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:49.975524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1085/2180] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:49 2022] DELETE /identity/v3/projects/8d10f59a27974c319846702d6b9a6944 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:49.980757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:49.987133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:50.016958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1096/2181] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:49 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:50.028759 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471d8b0-448a-44d7-a10c-a2802d9082a0 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:50.030320 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3471d8b0-448a-44d7-a10c-a2802d9082a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:50.088546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d01b2-52c1-4e02-94fd-426ff43788b2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.094464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.095359 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.095661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.095938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] PATH_INFO: `/v3/users/96deb8c34fee4c97ae12143fb64c510f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.098143 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96deb8c34fee4c97ae12143fb64c510f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.100190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:50.102017 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3471d8b0-448a-44d7-a10c-a2802d9082a0 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.105977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239946-d495-4e95-8999-03d02f0f11b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.106751 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239946-d495-4e95-8999-03d02f0f11b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.107116 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239946-d495-4e95-8999-03d02f0f11b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.107487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239946-d495-4e95-8999-03d02f0f11b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.109703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4f239946-d495-4e95-8999-03d02f0f11b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:50.150474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:50.172630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96deb8c34fee4c97ae12143fb64c510f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.185610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96deb8c34fee4c97ae12143fb64c510f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.187100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96deb8c34fee4c97ae12143fb64c510f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.200556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f239946-d495-4e95-8999-03d02f0f11b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.202120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f239946-d495-4e95-8999-03d02f0f11b2 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:50.202993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96deb8c34fee4c97ae12143fb64c510f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.214071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1097/2182] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:28:50.216540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96deb8c34fee4c97ae12143fb64c510f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.227478 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1086/2183] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:49 2022] DELETE /identity/v3/users/96deb8c34fee4c97ae12143fb64c510f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:28:50.693081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239946-d495-4e95-8999-03d02f0f11b2 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:50.706035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239946-d495-4e95-8999-03d02f0f11b2 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.755387 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:50.772814 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.773799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.774089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.774370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] PATH_INFO: `/v3/projects/80b39a505f1245f18c1ccd0b4117ce89` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.775648 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80b39a505f1245f18c1ccd0b4117ce89)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.777703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:50.831549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80b39a505f1245f18c1ccd0b4117ce89', 'request_id': 'req-733c0772-3890-48c5-8f36-d3bd870ac030'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.837795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80b39a505f1245f18c1ccd0b4117ce89', 'request_id': 'req-733c0772-3890-48c5-8f36-d3bd870ac030'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.846126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80b39a505f1245f18c1ccd0b4117ce89', 'request_id': 'req-733c0772-3890-48c5-8f36-d3bd870ac030'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.857235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] The token cache is being invalidate because project 80b39a505f1245f18c1ccd0b4117ce89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:50.857558 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.858219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:50.859795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1098/2184] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:50 2022] DELETE /identity/v3/projects/80b39a505f1245f18c1ccd0b4117ce89 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:50.864593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:50.866116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:50.882257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c4a933-9947-4a97-977e-c9d3d2bfaea0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:50.946414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:50.947567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:50.947855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:50.948170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] PATH_INFO: `/v3/users/d41011e36e0e420ca6b03edf26aa7ffa` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:50.949550 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d41011e36e0e420ca6b03edf26aa7ffa)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:50.951490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.003456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd41011e36e0e420ca6b03edf26aa7ffa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.010594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd41011e36e0e420ca6b03edf26aa7ffa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.011424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd41011e36e0e420ca6b03edf26aa7ffa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.023056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd41011e36e0e420ca6b03edf26aa7ffa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.031418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-6cafc10b-f7fd-40fb-ab40-e0b8fcb1b8c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd41011e36e0e420ca6b03edf26aa7ffa'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.038007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1087/2185] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:28:50 2022] DELETE /identity/v3/users/d41011e36e0e420ca6b03edf26aa7ffa => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:51.341687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.343804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:51.359921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c0772-3890-48c5-8f36-d3bd870ac030 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.438503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.440189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.440449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.440738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] PATH_INFO: `/v3/projects/251341d5d8e34cb1934b604972011b6e` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.442072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=251341d5d8e34cb1934b604972011b6e)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.444027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.499715 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '251341d5d8e34cb1934b604972011b6e', 'request_id': 'req-162843e3-7c7f-4a06-8afb-afcf9551e6a5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.508378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '251341d5d8e34cb1934b604972011b6e', 'request_id': 'req-162843e3-7c7f-4a06-8afb-afcf9551e6a5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.518246 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '251341d5d8e34cb1934b604972011b6e', 'request_id': 'req-162843e3-7c7f-4a06-8afb-afcf9551e6a5'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.529101 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] The token cache is being invalidate because project 251341d5d8e34cb1934b604972011b6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:28:51.529420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.530106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-162843e3-7c7f-4a06-8afb-afcf9551e6a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:28:51.531795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1099/2186] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:28:51 2022] DELETE /identity/v3/projects/251341d5d8e34cb1934b604972011b6e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:51.537514 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb2e00-8cc1-4c2f-b6ad-3ed6ce6b4893 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.538040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb2e00-8cc1-4c2f-b6ad-3ed6ce6b4893 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.538040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb2e00-8cc1-4c2f-b6ad-3ed6ce6b4893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.558560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-83cb2e00-8cc1-4c2f-b6ad-3ed6ce6b4893 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:51.559800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-83cb2e00-8cc1-4c2f-b6ad-3ed6ce6b4893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:51.591262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:51.647660 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1088/2187] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:51.658381 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f51e7-95bd-49cc-b4f4-e3a8e6014daf None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.658740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f51e7-95bd-49cc-b4f4-e3a8e6014daf None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.659079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f51e7-95bd-49cc-b4f4-e3a8e6014daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.667809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:51.683348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b05f51e7-95bd-49cc-b4f4-e3a8e6014daf None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:51.685539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b05f51e7-95bd-49cc-b4f4-e3a8e6014daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:51.695126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1100/2188] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:28:51.699305 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb2e00-8cc1-4c2f-b6ad-3ed6ce6b4893 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.715503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb2e00-8cc1-4c2f-b6ad-3ed6ce6b4893 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.723621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bd1f4-13af-4ce6-977c-a3bc59dbcbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.724416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bd1f4-13af-4ce6-977c-a3bc59dbcbdc admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.724920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bd1f4-13af-4ce6-977c-a3bc59dbcbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.725411 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941bd1f4-13af-4ce6-977c-a3bc59dbcbdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.726321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bd1f4-13af-4ce6-977c-a3bc59dbcbdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:28:51.726753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bd1f4-13af-4ce6-977c-a3bc59dbcbdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.728527 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941bd1f4-13af-4ce6-977c-a3bc59dbcbdc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.745047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1089/2189] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:28:51.756128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f51e7-95bd-49cc-b4f4-e3a8e6014daf None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.761883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f51e7-95bd-49cc-b4f4-e3a8e6014daf None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.775202 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1b1b9-1be9-460c-b619-cd383ec3359f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.775890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1b1b9-1be9-460c-b619-cd383ec3359f admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.776177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1b1b9-1be9-460c-b619-cd383ec3359f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.776456 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf1b1b9-1be9-460c-b619-cd383ec3359f admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.777173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1b1b9-1be9-460c-b619-cd383ec3359f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.779160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf1b1b9-1be9-460c-b619-cd383ec3359f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.812421 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1101/2190] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:28:51 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:51.818670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bd1f4-13af-4ce6-977c-a3bc59dbcbdc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.836898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941bd1f4-13af-4ce6-977c-a3bc59dbcbdc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.925577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894de4a-e2f9-4e95-8302-e06acd49f6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.927131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894de4a-e2f9-4e95-8302-e06acd49f6ac admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.927131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894de4a-e2f9-4e95-8302-e06acd49f6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.927131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1894de4a-e2f9-4e95-8302-e06acd49f6ac admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.927552 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894de4a-e2f9-4e95-8302-e06acd49f6ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.929475 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1894de4a-e2f9-4e95-8302-e06acd49f6ac admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.952424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1090/2191] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:28:51 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:28:51.961550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1b1b9-1be9-460c-b619-cd383ec3359f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:51.976018 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf1b1b9-1be9-460c-b619-cd383ec3359f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:51.981391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd63f3-1f39-40d3-883e-56ea3fa56c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:51.982365 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd63f3-1f39-40d3-883e-56ea3fa56c07 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:51.982649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd63f3-1f39-40d3-883e-56ea3fa56c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:51.982927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd63f3-1f39-40d3-883e-56ea3fa56c07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:51.983668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd63f3-1f39-40d3-883e-56ea3fa56c07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:51.985598 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd63f3-1f39-40d3-883e-56ea3fa56c07 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:51.998823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1102/2192] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:52.003616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894de4a-e2f9-4e95-8302-e06acd49f6ac admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.014155 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1894de4a-e2f9-4e95-8302-e06acd49f6ac admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.020640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce8fdbb-2756-477c-aeed-7cdcebb8ce4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.021320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8fdbb-2756-477c-aeed-7cdcebb8ce4c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.021638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8fdbb-2756-477c-aeed-7cdcebb8ce4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.021912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce8fdbb-2756-477c-aeed-7cdcebb8ce4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.022646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8fdbb-2756-477c-aeed-7cdcebb8ce4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.024248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce8fdbb-2756-477c-aeed-7cdcebb8ce4c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.035501 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1091/2193] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:28:52.039679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd63f3-1f39-40d3-883e-56ea3fa56c07 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.050253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd63f3-1f39-40d3-883e-56ea3fa56c07 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.055245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bf128-1c50-4f3f-bb02-840ebc52b47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.056138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bf128-1c50-4f3f-bb02-840ebc52b47d admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.056449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bf128-1c50-4f3f-bb02-840ebc52b47d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.056762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982bf128-1c50-4f3f-bb02-840ebc52b47d admin admin] PATH_INFO: `/v3/projects/cd13d03c37a049be9310f8819997b735/users/f40b9a23197b47058f4924478708c72b/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.074091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bf128-1c50-4f3f-bb02-840ebc52b47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd13d03c37a049be9310f8819997b735, user_id=f40b9a23197b47058f4924478708c72b, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.076785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982bf128-1c50-4f3f-bb02-840ebc52b47d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.091581 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1103/2194] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:28:52 2022] PUT /identity/v3/projects/cd13d03c37a049be9310f8819997b735/users/f40b9a23197b47058f4924478708c72b/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:28:52.105842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819a8a6-9c7c-4b10-b65f-9b9f6be161b5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.106124 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819a8a6-9c7c-4b10-b65f-9b9f6be161b5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.106417 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819a8a6-9c7c-4b10-b65f-9b9f6be161b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.138692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5819a8a6-9c7c-4b10-b65f-9b9f6be161b5 None None] MFA Rules not processed for user `f40b9a23197b47058f4924478708c72b`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:28:52.139859 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5819a8a6-9c7c-4b10-b65f-9b9f6be161b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:52.216488 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1092/2195] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:28:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:28:52.229785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bf128-1c50-4f3f-bb02-840ebc52b47d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:52.231773 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-982bf128-1c50-4f3f-bb02-840ebc52b47d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:52.296096 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982bf128-1c50-4f3f-bb02-840ebc52b47d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:52.300710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee554d-0175-4c21-865a-d26b8a8e61d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:52.301370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee554d-0175-4c21-865a-d26b8a8e61d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:52.301696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee554d-0175-4c21-865a-d26b8a8e61d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:52.301973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee554d-0175-4c21-865a-d26b8a8e61d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:52.311147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee554d-0175-4c21-865a-d26b8a8e61d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:52.312872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ee554d-0175-4c21-865a-d26b8a8e61d4 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:52.323385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1104/2196] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:28:54.866780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819a8a6-9c7c-4b10-b65f-9b9f6be161b5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:28:54.868277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5819a8a6-9c7c-4b10-b65f-9b9f6be161b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:54.886471 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819a8a6-9c7c-4b10-b65f-9b9f6be161b5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:28:54.890199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1ed081-ee58-4860-81eb-e66a5485e3af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:28:54.890860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed081-ee58-4860-81eb-e66a5485e3af service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:28:54.891157 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed081-ee58-4860-81eb-e66a5485e3af service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:28:54.891438 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1ed081-ee58-4860-81eb-e66a5485e3af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:28:54.893992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0e1ed081-ee58-4860-81eb-e66a5485e3af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:28:54.916487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:28:54.964499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ed081-ee58-4860-81eb-e66a5485e3af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:28:54.966008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1ed081-ee58-4860-81eb-e66a5485e3af service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:28:54.978725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1093/2197] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:28:54 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:02.318900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:04.974570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=20 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:14.078835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee554d-0175-4c21-865a-d26b8a8e61d4 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.151499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee554d-0175-4c21-865a-d26b8a8e61d4 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.156179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.157047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.157329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.157632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] PATH_INFO: `/v3/users/f40b9a23197b47058f4924478708c72b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.158820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f40b9a23197b47058f4924478708c72b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:14.161253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:14.210666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f40b9a23197b47058f4924478708c72b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:14.219002 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f40b9a23197b47058f4924478708c72b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:14.219843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f40b9a23197b47058f4924478708c72b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:14.234131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f40b9a23197b47058f4924478708c72b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:14.249047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f40b9a23197b47058f4924478708c72b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:14.256074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1105/2198] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:14 2022] DELETE /identity/v3/users/f40b9a23197b47058f4924478708c72b => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:14.260548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423fb17-7209-4220-ac5d-5219811aa59f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.261017 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423fb17-7209-4220-ac5d-5219811aa59f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.261114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6423fb17-7209-4220-ac5d-5219811aa59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.279877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6423fb17-7209-4220-ac5d-5219811aa59f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:14.281182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6423fb17-7209-4220-ac5d-5219811aa59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:14.354832 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1094/2199] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:14.367092 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:14.368383 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:14.446340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf0f278-ed47-4847-b1a1-cd078f42f449 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:14.453260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afde2cf-6b45-439e-8480-d45abc79c278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:14.453948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afde2cf-6b45-439e-8480-d45abc79c278 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:14.454234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afde2cf-6b45-439e-8480-d45abc79c278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:14.454517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afde2cf-6b45-439e-8480-d45abc79c278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:14.467982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afde2cf-6b45-439e-8480-d45abc79c278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:14.470091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afde2cf-6b45-439e-8480-d45abc79c278 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:14.484930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1106/2200] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:15.009273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423fb17-7209-4220-ac5d-5219811aa59f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.010883 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6423fb17-7209-4220-ac5d-5219811aa59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:15.041717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6423fb17-7209-4220-ac5d-5219811aa59f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.052702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.053713 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.053998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.054276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] PATH_INFO: `/v3/projects/cd13d03c37a049be9310f8819997b735` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.056773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd13d03c37a049be9310f8819997b735)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.064190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.145490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd13d03c37a049be9310f8819997b735', 'request_id': 'req-5cdd21e8-80c2-4422-96b1-7df7cffa6153'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.152172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd13d03c37a049be9310f8819997b735', 'request_id': 'req-5cdd21e8-80c2-4422-96b1-7df7cffa6153'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.161789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd13d03c37a049be9310f8819997b735', 'request_id': 'req-5cdd21e8-80c2-4422-96b1-7df7cffa6153'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.175827 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] The token cache is being invalidate because project cd13d03c37a049be9310f8819997b735 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:15.176258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.177132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5cdd21e8-80c2-4422-96b1-7df7cffa6153 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.178928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1095/2201] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:15 2022] DELETE /identity/v3/projects/cd13d03c37a049be9310f8819997b735 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:15.183892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec25f4-8cd3-4a85-a1a2-4d4b98aeef8a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.184425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec25f4-8cd3-4a85-a1a2-4d4b98aeef8a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.184785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dec25f4-8cd3-4a85-a1a2-4d4b98aeef8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.210537 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7dec25f4-8cd3-4a85-a1a2-4d4b98aeef8a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:15.213972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7dec25f4-8cd3-4a85-a1a2-4d4b98aeef8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:15.246149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:15.305138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1107/2202] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:15.375489 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c962-8324-438e-9295-69aeb685c9a5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.376049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c962-8324-438e-9295-69aeb685c9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.376049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c962-8324-438e-9295-69aeb685c9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.395871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a8f7c962-8324-438e-9295-69aeb685c9a5 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:15.397108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a8f7c962-8324-438e-9295-69aeb685c9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:15.405210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1096/2203] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:15.410461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec25f4-8cd3-4a85-a1a2-4d4b98aeef8a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.420485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dec25f4-8cd3-4a85-a1a2-4d4b98aeef8a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.425153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4429be-bfff-47ea-bc5f-c71e3e123116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.425841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4429be-bfff-47ea-bc5f-c71e3e123116 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.426128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4429be-bfff-47ea-bc5f-c71e3e123116 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.426405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4429be-bfff-47ea-bc5f-c71e3e123116 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.427176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4429be-bfff-47ea-bc5f-c71e3e123116 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:15.427483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4429be-bfff-47ea-bc5f-c71e3e123116 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.429125 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4429be-bfff-47ea-bc5f-c71e3e123116 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.441188 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1108/2204] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:15.445726 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7c962-8324-438e-9295-69aeb685c9a5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.454874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f7c962-8324-438e-9295-69aeb685c9a5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.462070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd71180-55fd-4e42-b174-fe3e7941561b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.462736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd71180-55fd-4e42-b174-fe3e7941561b admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.463018 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd71180-55fd-4e42-b174-fe3e7941561b admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.463292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd71180-55fd-4e42-b174-fe3e7941561b admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.463982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd71180-55fd-4e42-b174-fe3e7941561b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.465935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd71180-55fd-4e42-b174-fe3e7941561b admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.494291 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1097/2205] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:15 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:15.499064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4429be-bfff-47ea-bc5f-c71e3e123116 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.511601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4429be-bfff-47ea-bc5f-c71e3e123116 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.595075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fca123-e6f3-4afa-9f75-fcf24e0886e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.602278 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fca123-e6f3-4afa-9f75-fcf24e0886e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.602571 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fca123-e6f3-4afa-9f75-fcf24e0886e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.602839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fca123-e6f3-4afa-9f75-fcf24e0886e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.603568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fca123-e6f3-4afa-9f75-fcf24e0886e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.606060 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fca123-e6f3-4afa-9f75-fcf24e0886e9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.645179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd71180-55fd-4e42-b174-fe3e7941561b admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.650238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1109/2206] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:15 2022] POST /identity/v3/users => generated 299 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:15.650684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0cd71180-55fd-4e42-b174-fe3e7941561b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:15.651863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fca123-e6f3-4afa-9f75-fcf24e0886e9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.662719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fca123-e6f3-4afa-9f75-fcf24e0886e9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.670735 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd71180-55fd-4e42-b174-fe3e7941561b admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.670912 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f39fad-f803-4527-b168-0a0b5955f89d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.671473 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f39fad-f803-4527-b168-0a0b5955f89d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.671762 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f39fad-f803-4527-b168-0a0b5955f89d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.672043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f39fad-f803-4527-b168-0a0b5955f89d admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.672799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f39fad-f803-4527-b168-0a0b5955f89d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.674454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f39fad-f803-4527-b168-0a0b5955f89d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.678342 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.679503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.680022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.680621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] PATH_INFO: `/v3/users/fdb3c23f23e84f4cbf78871e8c192ec7` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.681988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fdb3c23f23e84f4cbf78871e8c192ec7)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.684006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.685794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1110/2207] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:29:15.689910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f39fad-f803-4527-b168-0a0b5955f89d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.703064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f39fad-f803-4527-b168-0a0b5955f89d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.710049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a8993-538a-414e-9c2e-c3f3e8ec2887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.710879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a8993-538a-414e-9c2e-c3f3e8ec2887 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.711271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a8993-538a-414e-9c2e-c3f3e8ec2887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.711654 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a8993-538a-414e-9c2e-c3f3e8ec2887 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.713584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a8993-538a-414e-9c2e-c3f3e8ec2887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.716383 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a8993-538a-414e-9c2e-c3f3e8ec2887 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.730832 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1111/2208] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:29:15.740931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a8993-538a-414e-9c2e-c3f3e8ec2887 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.759270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a8993-538a-414e-9c2e-c3f3e8ec2887 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:15.762153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fdb3c23f23e84f4cbf78871e8c192ec7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.765611 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:15.771685 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929d96c8-93be-43c2-90db-7e9a7e312eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:15.772579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d96c8-93be-43c2-90db-7e9a7e312eb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.772874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d96c8-93be-43c2-90db-7e9a7e312eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.773254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929d96c8-93be-43c2-90db-7e9a7e312eb8 admin admin] PATH_INFO: `/v3/projects/4f1801764ad8413cb57728a1277c0ea5/users/17d17c97e4f64c898e63f0546465f6f3/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.778462 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fdb3c23f23e84f4cbf78871e8c192ec7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.780135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fdb3c23f23e84f4cbf78871e8c192ec7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.794862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fdb3c23f23e84f4cbf78871e8c192ec7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.803170 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d96c8-93be-43c2-90db-7e9a7e312eb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f1801764ad8413cb57728a1277c0ea5, user_id=17d17c97e4f64c898e63f0546465f6f3, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:15.807028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2f6eb039-28aa-42da-8ec1-5c9723d9d456 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fdb3c23f23e84f4cbf78871e8c192ec7'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:15.810072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929d96c8-93be-43c2-90db-7e9a7e312eb8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:15.818101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1098/2209] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:15 2022] DELETE /identity/v3/users/fdb3c23f23e84f4cbf78871e8c192ec7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:15.822782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266b5ac-1c3d-409e-8b44-3127b4d98616 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.823100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266b5ac-1c3d-409e-8b44-3127b4d98616 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.823380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2266b5ac-1c3d-409e-8b44-3127b4d98616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.838207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1112/2210] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:15 2022] PUT /identity/v3/projects/4f1801764ad8413cb57728a1277c0ea5/users/17d17c97e4f64c898e63f0546465f6f3/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:15.848052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79debd-2236-4c77-a98f-bc20d3b4fabe None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:15.848339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79debd-2236-4c77-a98f-bc20d3b4fabe None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:15.848614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd79debd-2236-4c77-a98f-bc20d3b4fabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:15.849363 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2266b5ac-1c3d-409e-8b44-3127b4d98616 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:15.850743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2266b5ac-1c3d-409e-8b44-3127b4d98616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:15.892737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bd79debd-2236-4c77-a98f-bc20d3b4fabe None None] MFA Rules not processed for user `17d17c97e4f64c898e63f0546465f6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:15.896561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bd79debd-2236-4c77-a98f-bc20d3b4fabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:15.950392 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1099/2211] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:15.961466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266b5ac-1c3d-409e-8b44-3127b4d98616 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:15.963890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2266b5ac-1c3d-409e-8b44-3127b4d98616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:15.965882 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:16.000417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1113/2212] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:16.015329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79debd-2236-4c77-a98f-bc20d3b4fabe None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.017475 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bd79debd-2236-4c77-a98f-bc20d3b4fabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:16.062658 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2266b5ac-1c3d-409e-8b44-3127b4d98616 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.071649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1149406-4f8a-4923-918a-7e13c931049d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.072720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1149406-4f8a-4923-918a-7e13c931049d service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.073084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1149406-4f8a-4923-918a-7e13c931049d service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.073480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1149406-4f8a-4923-918a-7e13c931049d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.091621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1149406-4f8a-4923-918a-7e13c931049d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.093204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1149406-4f8a-4923-918a-7e13c931049d service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.120228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1100/2213] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:16.138611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd79debd-2236-4c77-a98f-bc20d3b4fabe None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.142698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23d20f3-7bb2-45fc-8adf-6fa9bdedacb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.143369 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d20f3-7bb2-45fc-8adf-6fa9bdedacb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.143659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d20f3-7bb2-45fc-8adf-6fa9bdedacb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.144926 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23d20f3-7bb2-45fc-8adf-6fa9bdedacb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.156297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d20f3-7bb2-45fc-8adf-6fa9bdedacb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.158015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23d20f3-7bb2-45fc-8adf-6fa9bdedacb9 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.170870 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1114/2214] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:16 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:29:16.819398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1149406-4f8a-4923-918a-7e13c931049d service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:16.821597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d1149406-4f8a-4923-918a-7e13c931049d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:16.845653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1149406-4f8a-4923-918a-7e13c931049d service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:16.867137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:16.868028 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:16.868323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:16.868669 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] PATH_INFO: `/v3/projects/1343502925e4438c87f823fc530a5a58` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:16.871822 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1343502925e4438c87f823fc530a5a58)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:16.875472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:16.981425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1343502925e4438c87f823fc530a5a58', 'request_id': 'req-7d4f7710-0500-40c8-b782-1f411871a93a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:16.995796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1343502925e4438c87f823fc530a5a58', 'request_id': 'req-7d4f7710-0500-40c8-b782-1f411871a93a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:17.016399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1343502925e4438c87f823fc530a5a58', 'request_id': 'req-7d4f7710-0500-40c8-b782-1f411871a93a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:17.018634 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:17.051582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] The token cache is being invalidate because project 1343502925e4438c87f823fc530a5a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:17.052443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:17.053519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d4f7710-0500-40c8-b782-1f411871a93a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:17.055534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1101/2215] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:16 2022] DELETE /identity/v3/projects/1343502925e4438c87f823fc530a5a58 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:17.062113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd609373-2e64-488b-b337-6ccd158a8cb7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.063743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd609373-2e64-488b-b337-6ccd158a8cb7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.064190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd609373-2e64-488b-b337-6ccd158a8cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.092321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-cd609373-2e64-488b-b337-6ccd158a8cb7 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:17.093775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-cd609373-2e64-488b-b337-6ccd158a8cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:17.160396 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1115/2216] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:17.168388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e596f8-d4b5-451c-9aaa-84823d295e89 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.169502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e596f8-d4b5-451c-9aaa-84823d295e89 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.169849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e596f8-d4b5-451c-9aaa-84823d295e89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.194457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-99e596f8-d4b5-451c-9aaa-84823d295e89 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:17.195709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-99e596f8-d4b5-451c-9aaa-84823d295e89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:17.203924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1102/2217] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:17.208378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd609373-2e64-488b-b337-6ccd158a8cb7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.218291 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd609373-2e64-488b-b337-6ccd158a8cb7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.224684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e2959-d146-42c4-aef9-fa5e3e20c348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.225352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e2959-d146-42c4-aef9-fa5e3e20c348 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.225724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e2959-d146-42c4-aef9-fa5e3e20c348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.225963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e2959-d146-42c4-aef9-fa5e3e20c348 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.226712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e2959-d146-42c4-aef9-fa5e3e20c348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:17.227028 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e2959-d146-42c4-aef9-fa5e3e20c348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.228657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e2959-d146-42c4-aef9-fa5e3e20c348 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.245767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1116/2218] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:17.247040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e596f8-d4b5-451c-9aaa-84823d295e89 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.262524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e596f8-d4b5-451c-9aaa-84823d295e89 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.267685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b91de-6c63-48df-9144-22d772b66fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.268557 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b91de-6c63-48df-9144-22d772b66fa6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.269054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b91de-6c63-48df-9144-22d772b66fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.269479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b91de-6c63-48df-9144-22d772b66fa6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.270289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b91de-6c63-48df-9144-22d772b66fa6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.272387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965b91de-6c63-48df-9144-22d772b66fa6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.308766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1103/2219] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:17 2022] POST /identity/v3/projects => generated 381 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:29:17.313503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e2959-d146-42c4-aef9-fa5e3e20c348 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.325789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e2959-d146-42c4-aef9-fa5e3e20c348 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.415693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f9e9b-a867-4de3-a2df-66b803019483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.416440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f9e9b-a867-4de3-a2df-66b803019483 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.417495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f9e9b-a867-4de3-a2df-66b803019483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.418463 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26f9e9b-a867-4de3-a2df-66b803019483 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.419388 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f9e9b-a867-4de3-a2df-66b803019483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.422698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26f9e9b-a867-4de3-a2df-66b803019483 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.450191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1117/2220] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:17 2022] POST /identity/v3/users => generated 291 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:17.454259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b91de-6c63-48df-9144-22d772b66fa6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.466978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b91de-6c63-48df-9144-22d772b66fa6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.472843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c8228-5877-431b-8792-6ba2651b3ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.474019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c8228-5877-431b-8792-6ba2651b3ca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.474019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c8228-5877-431b-8792-6ba2651b3ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.474218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01c8228-5877-431b-8792-6ba2651b3ca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.475090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c8228-5877-431b-8792-6ba2651b3ca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.476758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01c8228-5877-431b-8792-6ba2651b3ca0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.488793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1104/2221] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:29:17.492735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f9e9b-a867-4de3-a2df-66b803019483 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.499769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:17.505681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26f9e9b-a867-4de3-a2df-66b803019483 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.516786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea127d-7cc9-492f-becc-b8219862f0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.518102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea127d-7cc9-492f-becc-b8219862f0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.518556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea127d-7cc9-492f-becc-b8219862f0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.519086 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea127d-7cc9-492f-becc-b8219862f0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.520388 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea127d-7cc9-492f-becc-b8219862f0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.522221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea127d-7cc9-492f-becc-b8219862f0b4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.543652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1118/2222] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:29:17.547628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c8228-5877-431b-8792-6ba2651b3ca0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.563024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01c8228-5877-431b-8792-6ba2651b3ca0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.569003 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af060ef-2012-4956-837d-824e23c021e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.571318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af060ef-2012-4956-837d-824e23c021e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.571689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af060ef-2012-4956-837d-824e23c021e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.572043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af060ef-2012-4956-837d-824e23c021e3 admin admin] PATH_INFO: `/v3/projects/86acb815cd62409d9856563af3037d38/users/d582e9b7a49c4ea6b9a1d6986d6f4163/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.592942 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af060ef-2012-4956-837d-824e23c021e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86acb815cd62409d9856563af3037d38, user_id=d582e9b7a49c4ea6b9a1d6986d6f4163, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.595860 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af060ef-2012-4956-837d-824e23c021e3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.620441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1105/2223] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:17 2022] PUT /identity/v3/projects/86acb815cd62409d9856563af3037d38/users/d582e9b7a49c4ea6b9a1d6986d6f4163/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:29:17.625684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c62b9-3ac1-452b-ac62-f3d9dab0919c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.626242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c62b9-3ac1-452b-ac62-f3d9dab0919c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.626597 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0c62b9-3ac1-452b-ac62-f3d9dab0919c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.658855 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0b0c62b9-3ac1-452b-ac62-f3d9dab0919c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:17.659896 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0b0c62b9-3ac1-452b-ac62-f3d9dab0919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:17.752522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1119/2224] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:17.762306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af060ef-2012-4956-837d-824e23c021e3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:17.764442 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0af060ef-2012-4956-837d-824e23c021e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:17.857664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af060ef-2012-4956-837d-824e23c021e3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:17.861971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e38ee8-8ac1-48f6-8373-96bde6d7b7b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:17.862688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e38ee8-8ac1-48f6-8373-96bde6d7b7b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.863101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e38ee8-8ac1-48f6-8373-96bde6d7b7b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.863426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e38ee8-8ac1-48f6-8373-96bde6d7b7b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.874016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e38ee8-8ac1-48f6-8373-96bde6d7b7b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:17.875602 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e38ee8-8ac1-48f6-8373-96bde6d7b7b3 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:17.881212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473864-54d6-464f-93a9-58c720d2ff44 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.882104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473864-54d6-464f-93a9-58c720d2ff44 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.882924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe473864-54d6-464f-93a9-58c720d2ff44 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.884445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1120/2225] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:29:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 12 13:29:17.887353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1106/2226] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:29:17.896100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e050d9-3822-4563-8d2a-1c13516b69db None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:17.896516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e050d9-3822-4563-8d2a-1c13516b69db None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:17.896807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e050d9-3822-4563-8d2a-1c13516b69db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:17.925087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e8e050d9-3822-4563-8d2a-1c13516b69db None None] MFA Rules not processed for user `2e97119806774c2482a9886b11ab6f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:17.926728 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e8e050d9-3822-4563-8d2a-1c13516b69db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:18.051857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1121/2227] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:29:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:18.191100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e38ee8-8ac1-48f6-8373-96bde6d7b7b3 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.194796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c8e38ee8-8ac1-48f6-8373-96bde6d7b7b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:18.217172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e38ee8-8ac1-48f6-8373-96bde6d7b7b3 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.227239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f59a38-43f1-4295-bf81-4bd06142d287 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e97119806774c2482a9886b11ab6f70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.228464 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f59a38-43f1-4295-bf81-4bd06142d287 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.229030 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f59a38-43f1-4295-bf81-4bd06142d287 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.229628 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f59a38-43f1-4295-bf81-4bd06142d287 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.252402 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e050d9-3822-4563-8d2a-1c13516b69db None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.253975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e8e050d9-3822-4563-8d2a-1c13516b69db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:18.258110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f59a38-43f1-4295-bf81-4bd06142d287 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.259980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f59a38-43f1-4295-bf81-4bd06142d287 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.274262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1107/2228] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:29:18.291863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e050d9-3822-4563-8d2a-1c13516b69db None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.300713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.317750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.318098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.318408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] PATH_INFO: `/v3/users/83bbc8ad40b44661b526511ecbc51027` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.319883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=83bbc8ad40b44661b526511ecbc51027)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.322394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.431887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83bbc8ad40b44661b526511ecbc51027'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.440273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83bbc8ad40b44661b526511ecbc51027'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.441758 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83bbc8ad40b44661b526511ecbc51027'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.464627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83bbc8ad40b44661b526511ecbc51027'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.489546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-7e5a0736-a23d-41d3-9cbe-f06afe1387cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83bbc8ad40b44661b526511ecbc51027'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.493957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:18.510157 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1122/2229] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:18 2022] DELETE /identity/v3/users/83bbc8ad40b44661b526511ecbc51027 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:18.698388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f59a38-43f1-4295-bf81-4bd06142d287 service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:18.712354 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f59a38-43f1-4295-bf81-4bd06142d287 service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:18.792326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:18.793959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.794251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.794532 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] PATH_INFO: `/v3/projects/d44719e63bb947bdb13e9309d1576521` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.796005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d44719e63bb947bdb13e9309d1576521)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:18.798022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:18.824766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:18.855901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd44719e63bb947bdb13e9309d1576521', 'request_id': 'req-7f2dcb75-3329-4820-9637-36d58682bf7d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.864022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd44719e63bb947bdb13e9309d1576521', 'request_id': 'req-7f2dcb75-3329-4820-9637-36d58682bf7d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.873431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd44719e63bb947bdb13e9309d1576521', 'request_id': 'req-7f2dcb75-3329-4820-9637-36d58682bf7d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.890974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] The token cache is being invalidate because project d44719e63bb947bdb13e9309d1576521 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:18.891125 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.891814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7f2dcb75-3329-4820-9637-36d58682bf7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:18.893540 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1108/2230] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:18 2022] DELETE /identity/v3/projects/d44719e63bb947bdb13e9309d1576521 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:18.899847 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b3327-6a36-4028-ac5e-0f66ae288161 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:18.900191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b3327-6a36-4028-ac5e-0f66ae288161 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:18.900478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259b3327-6a36-4028-ac5e-0f66ae288161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:18.922474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-259b3327-6a36-4028-ac5e-0f66ae288161 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:18.923883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-259b3327-6a36-4028-ac5e-0f66ae288161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:18.991813 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1123/2231] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:19.001216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d81ae6-3724-44a4-9065-1babefb9e83a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.010012 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d81ae6-3724-44a4-9065-1babefb9e83a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.011439 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d81ae6-3724-44a4-9065-1babefb9e83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.032039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a5d81ae6-3724-44a4-9065-1babefb9e83a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:19.033398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a5d81ae6-3724-44a4-9065-1babefb9e83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:19.042067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1109/2232] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:19.046469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b3327-6a36-4028-ac5e-0f66ae288161 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.057993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259b3327-6a36-4028-ac5e-0f66ae288161 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.063084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528256f-7886-4ab6-a20b-67ab4351160e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.063843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528256f-7886-4ab6-a20b-67ab4351160e admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.064199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528256f-7886-4ab6-a20b-67ab4351160e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.064548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528256f-7886-4ab6-a20b-67ab4351160e admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.065441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528256f-7886-4ab6-a20b-67ab4351160e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:19.066578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528256f-7886-4ab6-a20b-67ab4351160e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.068318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528256f-7886-4ab6-a20b-67ab4351160e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.084549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1124/2233] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:19.085817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d81ae6-3724-44a4-9065-1babefb9e83a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.095694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d81ae6-3724-44a4-9065-1babefb9e83a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.101004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f11461-b40b-4ffd-ad19-61e8af559162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.101779 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f11461-b40b-4ffd-ad19-61e8af559162 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.102065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f11461-b40b-4ffd-ad19-61e8af559162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.102342 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f11461-b40b-4ffd-ad19-61e8af559162 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.103061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f11461-b40b-4ffd-ad19-61e8af559162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.104970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f11461-b40b-4ffd-ad19-61e8af559162 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.134377 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1110/2234] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:19 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:19.138891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528256f-7886-4ab6-a20b-67ab4351160e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.149414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528256f-7886-4ab6-a20b-67ab4351160e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.209209 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1ab9-d016-4703-bad4-6c94208eae23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.209923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1ab9-d016-4703-bad4-6c94208eae23 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.210207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1ab9-d016-4703-bad4-6c94208eae23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.210483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10b1ab9-d016-4703-bad4-6c94208eae23 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.211179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b1ab9-d016-4703-bad4-6c94208eae23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.213111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10b1ab9-d016-4703-bad4-6c94208eae23 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.237892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1125/2235] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:19 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:19.242362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f11461-b40b-4ffd-ad19-61e8af559162 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.257087 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f11461-b40b-4ffd-ad19-61e8af559162 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.263632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df0f0a-3fad-4b98-88bf-fd2088a3f1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.264881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df0f0a-3fad-4b98-88bf-fd2088a3f1bf admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.264881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df0f0a-3fad-4b98-88bf-fd2088a3f1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.265567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df0f0a-3fad-4b98-88bf-fd2088a3f1bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.266330 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df0f0a-3fad-4b98-88bf-fd2088a3f1bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.268276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df0f0a-3fad-4b98-88bf-fd2088a3f1bf admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.283179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1111/2236] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:19.285205 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1ab9-d016-4703-bad4-6c94208eae23 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.306220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10b1ab9-d016-4703-bad4-6c94208eae23 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.312494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee34322c-c19b-4f33-8808-bec70c65d75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.317049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34322c-c19b-4f33-8808-bec70c65d75d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.318262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34322c-c19b-4f33-8808-bec70c65d75d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.318742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee34322c-c19b-4f33-8808-bec70c65d75d admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.320502 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34322c-c19b-4f33-8808-bec70c65d75d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.323354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee34322c-c19b-4f33-8808-bec70c65d75d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.344623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1126/2237] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:19.346175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df0f0a-3fad-4b98-88bf-fd2088a3f1bf admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.348569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec8e2f-efed-425a-8b10-98fce81e0601 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.349131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec8e2f-efed-425a-8b10-98fce81e0601 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.349566 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec8e2f-efed-425a-8b10-98fce81e0601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.360480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df0f0a-3fad-4b98-88bf-fd2088a3f1bf admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.367946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16997f9d-1327-4cc4-adee-6e8d388448d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.368966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16997f9d-1327-4cc4-adee-6e8d388448d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.369329 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16997f9d-1327-4cc4-adee-6e8d388448d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.369709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16997f9d-1327-4cc4-adee-6e8d388448d8 admin admin] PATH_INFO: `/v3/projects/763c7470856b43ee8b139b9050fa360a/users/5bfe77d7894b4cc897f18137fa317ddc/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.392027 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16997f9d-1327-4cc4-adee-6e8d388448d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=763c7470856b43ee8b139b9050fa360a, user_id=5bfe77d7894b4cc897f18137fa317ddc, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.397123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16997f9d-1327-4cc4-adee-6e8d388448d8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.398897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-51ec8e2f-efed-425a-8b10-98fce81e0601 None None] MFA Rules not processed for user `d582e9b7a49c4ea6b9a1d6986d6f4163`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:19.404549 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-51ec8e2f-efed-425a-8b10-98fce81e0601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:19.421913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1112/2238] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:19 2022] PUT /identity/v3/projects/763c7470856b43ee8b139b9050fa360a/users/5bfe77d7894b4cc897f18137fa317ddc/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:19.432404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e31d5-53a2-4009-b4ce-fe50213be8fd None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.432696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e31d5-53a2-4009-b4ce-fe50213be8fd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.432978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126e31d5-53a2-4009-b4ce-fe50213be8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.467458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-126e31d5-53a2-4009-b4ce-fe50213be8fd None None] MFA Rules not processed for user `5bfe77d7894b4cc897f18137fa317ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:19.469001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-126e31d5-53a2-4009-b4ce-fe50213be8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:19.500447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:19.521503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1127/2239] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:19.529809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec8e2f-efed-425a-8b10-98fce81e0601 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.531774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-51ec8e2f-efed-425a-8b10-98fce81e0601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:19.563809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1113/2240] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:19.577930 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e31d5-53a2-4009-b4ce-fe50213be8fd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.582574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-126e31d5-53a2-4009-b4ce-fe50213be8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:19.624064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec8e2f-efed-425a-8b10-98fce81e0601 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.624595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:19.667387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126e31d5-53a2-4009-b4ce-fe50213be8fd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:19.675898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa50bb-657d-4a31-8a97-a451148230ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.677018 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa50bb-657d-4a31-8a97-a451148230ff service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.677322 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa50bb-657d-4a31-8a97-a451148230ff service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.677633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa50bb-657d-4a31-8a97-a451148230ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.682392 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917537d-d7c6-402e-888e-b9c13946a6b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:19.683131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917537d-d7c6-402e-888e-b9c13946a6b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:19.683489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917537d-d7c6-402e-888e-b9c13946a6b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:19.683838 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917537d-d7c6-402e-888e-b9c13946a6b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:19.688251 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa50bb-657d-4a31-8a97-a451148230ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.690567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa50bb-657d-4a31-8a97-a451148230ff service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.702680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1114/2241] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:19.753899 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917537d-d7c6-402e-888e-b9c13946a6b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:19.755470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917537d-d7c6-402e-888e-b9c13946a6b2 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:19.766759 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1128/2242] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:19.892005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa50bb-657d-4a31-8a97-a451148230ff service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:19.893482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fffa50bb-657d-4a31-8a97-a451148230ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:20.007254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa50bb-657d-4a31-8a97-a451148230ff service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.015294 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.016649 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.017130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.017677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] PATH_INFO: `/v3/users/0013c27e2a2b440cb6a57bad2a338398` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.019907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=0013c27e2a2b440cb6a57bad2a338398)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:20.022844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:20.137429 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0013c27e2a2b440cb6a57bad2a338398'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.154356 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0013c27e2a2b440cb6a57bad2a338398'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.155701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0013c27e2a2b440cb6a57bad2a338398'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.175682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0013c27e2a2b440cb6a57bad2a338398'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.195965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0013c27e2a2b440cb6a57bad2a338398'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.203559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1115/2243] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:19 2022] DELETE /identity/v3/users/0013c27e2a2b440cb6a57bad2a338398 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:20.422783 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917537d-d7c6-402e-888e-b9c13946a6b2 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.442953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917537d-d7c6-402e-888e-b9c13946a6b2 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.512936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.513842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.514129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.514407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] PATH_INFO: `/v3/projects/a7bf50e110e740a2900febabade6f840` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.515619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7bf50e110e740a2900febabade6f840)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:20.517576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:20.528110 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:20.579119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7bf50e110e740a2900febabade6f840', 'request_id': 'req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.586577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7bf50e110e740a2900febabade6f840', 'request_id': 'req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.594833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7bf50e110e740a2900febabade6f840', 'request_id': 'req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.606242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] The token cache is being invalidate because project a7bf50e110e740a2900febabade6f840 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:20.606641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.607161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.608700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1129/2244] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:20 2022] DELETE /identity/v3/projects/a7bf50e110e740a2900febabade6f840 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:29:20.613536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.615063 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:20.631850 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ab90a-c36e-434c-bc33-d11286c37dae admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:20.670290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:20.692261 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:20.693136 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:20.693458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:20.693736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] PATH_INFO: `/v3/users/b747ba8ddead4b788b8c4a33b2df96b5` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:20.694939 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b747ba8ddead4b788b8c4a33b2df96b5)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:20.696836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:20.749285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b747ba8ddead4b788b8c4a33b2df96b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.756776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b747ba8ddead4b788b8c4a33b2df96b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.757490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b747ba8ddead4b788b8c4a33b2df96b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.770495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b747ba8ddead4b788b8c4a33b2df96b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.777815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-883ac467-07f5-48a8-ba52-6540dea3d2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b747ba8ddead4b788b8c4a33b2df96b5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:20.784042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1116/2245] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:20 2022] DELETE /identity/v3/users/b747ba8ddead4b788b8c4a33b2df96b5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:20.961726 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:20.963483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:20.977469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91136cb-dbf1-4214-afee-fdcf1f7dbc25 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.043444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.044326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.044617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.044899 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] PATH_INFO: `/v3/projects/db6f65dcd8cf4e179457181d75ff2f75` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.046543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db6f65dcd8cf4e179457181d75ff2f75)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.048507 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.098418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db6f65dcd8cf4e179457181d75ff2f75', 'request_id': 'req-0510e73f-15ab-49bf-a4be-0c87008e4a29'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:21.104543 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db6f65dcd8cf4e179457181d75ff2f75', 'request_id': 'req-0510e73f-15ab-49bf-a4be-0c87008e4a29'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:21.111909 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db6f65dcd8cf4e179457181d75ff2f75', 'request_id': 'req-0510e73f-15ab-49bf-a4be-0c87008e4a29'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:21.123248 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] The token cache is being invalidate because project db6f65dcd8cf4e179457181d75ff2f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:21.123538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:21.124181 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0510e73f-15ab-49bf-a4be-0c87008e4a29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:21.125767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1130/2246] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:20 2022] DELETE /identity/v3/projects/db6f65dcd8cf4e179457181d75ff2f75 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:21.131223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72d2c-8854-4585-a697-1785cf92ea9f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.131723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72d2c-8854-4585-a697-1785cf92ea9f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.131813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc72d2c-8854-4585-a697-1785cf92ea9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.155226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2fc72d2c-8854-4585-a697-1785cf92ea9f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:21.156909 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2fc72d2c-8854-4585-a697-1785cf92ea9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:21.242842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1117/2247] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:21.250923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b3616-83d7-4f4c-afd1-e8c1e77d97e5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.250923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b3616-83d7-4f4c-afd1-e8c1e77d97e5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.251221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410b3616-83d7-4f4c-afd1-e8c1e77d97e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.273118 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-410b3616-83d7-4f4c-afd1-e8c1e77d97e5 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:21.274371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-410b3616-83d7-4f4c-afd1-e8c1e77d97e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:21.282240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1131/2248] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:21.286542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc72d2c-8854-4585-a697-1785cf92ea9f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.296420 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc72d2c-8854-4585-a697-1785cf92ea9f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.301890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a311-ee2a-43e6-8b46-4121599cf22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.302549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a311-ee2a-43e6-8b46-4121599cf22a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.302835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a311-ee2a-43e6-8b46-4121599cf22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.303116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e30a311-ee2a-43e6-8b46-4121599cf22a admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.303849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30a311-ee2a-43e6-8b46-4121599cf22a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:21.304160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30a311-ee2a-43e6-8b46-4121599cf22a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.305816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e30a311-ee2a-43e6-8b46-4121599cf22a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.318830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1118/2249] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:21.323310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b3616-83d7-4f4c-afd1-e8c1e77d97e5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.333612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410b3616-83d7-4f4c-afd1-e8c1e77d97e5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.338366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e1e67-dd0e-4511-84d0-0b91f04f3bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.338999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e1e67-dd0e-4511-84d0-0b91f04f3bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.339286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e1e67-dd0e-4511-84d0-0b91f04f3bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.339561 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12e1e67-dd0e-4511-84d0-0b91f04f3bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.340261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e1e67-dd0e-4511-84d0-0b91f04f3bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.342201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12e1e67-dd0e-4511-84d0-0b91f04f3bb0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.363026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:21.371599 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1132/2250] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:21 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:21.375130 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a311-ee2a-43e6-8b46-4121599cf22a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.386277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e30a311-ee2a-43e6-8b46-4121599cf22a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.430474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:21.445720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c910b0-8308-4959-9037-658f39af2028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.446402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c910b0-8308-4959-9037-658f39af2028 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.446688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c910b0-8308-4959-9037-658f39af2028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.446966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c910b0-8308-4959-9037-658f39af2028 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.447674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c910b0-8308-4959-9037-658f39af2028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.449627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c910b0-8308-4959-9037-658f39af2028 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.471353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1119/2251] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:21 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:21.476076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e1e67-dd0e-4511-84d0-0b91f04f3bb0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.487005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12e1e67-dd0e-4511-84d0-0b91f04f3bb0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.492522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7b0d1-055c-486f-8b53-5156432f1a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.493206 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7b0d1-055c-486f-8b53-5156432f1a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.493546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7b0d1-055c-486f-8b53-5156432f1a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.493833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7b0d1-055c-486f-8b53-5156432f1a79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.494852 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7b0d1-055c-486f-8b53-5156432f1a79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.497289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7b0d1-055c-486f-8b53-5156432f1a79 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.515299 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1133/2252] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:21.519542 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c910b0-8308-4959-9037-658f39af2028 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.531398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c910b0-8308-4959-9037-658f39af2028 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.538764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8a4864-75b6-4657-a830-51bfb60335be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.539682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a4864-75b6-4657-a830-51bfb60335be admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.540024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a4864-75b6-4657-a830-51bfb60335be admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.540421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8a4864-75b6-4657-a830-51bfb60335be admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.541301 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a4864-75b6-4657-a830-51bfb60335be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.543276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8a4864-75b6-4657-a830-51bfb60335be admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.555229 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1120/2253] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:21.559444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7b0d1-055c-486f-8b53-5156432f1a79 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.571256 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7b0d1-055c-486f-8b53-5156432f1a79 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.577827 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258064f-188d-456a-86a4-8ec4db847e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.579221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258064f-188d-456a-86a4-8ec4db847e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.579573 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258064f-188d-456a-86a4-8ec4db847e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.580062 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258064f-188d-456a-86a4-8ec4db847e57 admin admin] PATH_INFO: `/v3/projects/13d7dac7e7b74443864ce5192d18ee20/users/9a2d44c86e2648f4820c8d9c411d949a/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.603591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258064f-188d-456a-86a4-8ec4db847e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13d7dac7e7b74443864ce5192d18ee20, user_id=9a2d44c86e2648f4820c8d9c411d949a, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.606707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258064f-188d-456a-86a4-8ec4db847e57 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.629872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1134/2254] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:21 2022] PUT /identity/v3/projects/13d7dac7e7b74443864ce5192d18ee20/users/9a2d44c86e2648f4820c8d9c411d949a/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:21.635789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1268b-acd2-49fd-9747-dd731ce4afff None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.636081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1268b-acd2-49fd-9747-dd731ce4afff None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.636445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1268b-acd2-49fd-9747-dd731ce4afff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.668623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bfc1268b-acd2-49fd-9747-dd731ce4afff None None] MFA Rules not processed for user `9a2d44c86e2648f4820c8d9c411d949a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:21.670040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bfc1268b-acd2-49fd-9747-dd731ce4afff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:21.748582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1121/2255] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:21.760577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258064f-188d-456a-86a4-8ec4db847e57 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.762493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c258064f-188d-456a-86a4-8ec4db847e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:21.822252 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258064f-188d-456a-86a4-8ec4db847e57 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.825902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e94d2d3-dc45-479c-935f-01473a970463 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.826546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d2d3-dc45-479c-935f-01473a970463 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.826834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d2d3-dc45-479c-935f-01473a970463 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.827112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e94d2d3-dc45-479c-935f-01473a970463 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.835781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94d2d3-dc45-479c-935f-01473a970463 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.837222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e94d2d3-dc45-479c-935f-01473a970463 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:21.846098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1135/2256] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:21.866372 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1268b-acd2-49fd-9747-dd731ce4afff None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:21.952082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1268b-acd2-49fd-9747-dd731ce4afff None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:21.956616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:21.957634 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:21.957921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:21.958198 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] PATH_INFO: `/v3/users/9a2d44c86e2648f4820c8d9c411d949a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:21.959422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a2d44c86e2648f4820c8d9c411d949a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:21.961351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.012685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a2d44c86e2648f4820c8d9c411d949a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.020722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a2d44c86e2648f4820c8d9c411d949a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.021517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a2d44c86e2648f4820c8d9c411d949a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.022692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:22.037756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a2d44c86e2648f4820c8d9c411d949a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.045479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a2d44c86e2648f4820c8d9c411d949a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.052615 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1122/2257] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:21 2022] DELETE /identity/v3/users/9a2d44c86e2648f4820c8d9c411d949a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:22.057025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0f703-2834-40ad-92f7-c730d63f28fe None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.057316 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0f703-2834-40ad-92f7-c730d63f28fe None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.057622 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0f703-2834-40ad-92f7-c730d63f28fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.075709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c5f0f703-2834-40ad-92f7-c730d63f28fe None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:22.076979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c5f0f703-2834-40ad-92f7-c730d63f28fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:22.137025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1136/2258] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:22.148352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.149842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:22.209316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21bfc69-eef2-4a6a-9fc6-b6760680e5a4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.213591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ea0e7c-f781-407f-863d-82a62f7577ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.214396 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0e7c-f781-407f-863d-82a62f7577ea service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.214689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0e7c-f781-407f-863d-82a62f7577ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.214969 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea0e7c-f781-407f-863d-82a62f7577ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.230526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea0e7c-f781-407f-863d-82a62f7577ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.231987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ea0e7c-f781-407f-863d-82a62f7577ea service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.244934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1123/2259] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:22.538492 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0f703-2834-40ad-92f7-c730d63f28fe None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.540133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c5f0f703-2834-40ad-92f7-c730d63f28fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:22.554319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0f703-2834-40ad-92f7-c730d63f28fe None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.559265 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.560323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.560724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.561069 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] PATH_INFO: `/v3/projects/13d7dac7e7b74443864ce5192d18ee20` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.562596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13d7dac7e7b74443864ce5192d18ee20)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.564990 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.616981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13d7dac7e7b74443864ce5192d18ee20', 'request_id': 'req-8d22eece-ef39-4830-9513-72e9a577cb60'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.623520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13d7dac7e7b74443864ce5192d18ee20', 'request_id': 'req-8d22eece-ef39-4830-9513-72e9a577cb60'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.632214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13d7dac7e7b74443864ce5192d18ee20', 'request_id': 'req-8d22eece-ef39-4830-9513-72e9a577cb60'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.633761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:22.647874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] The token cache is being invalidate because project 13d7dac7e7b74443864ce5192d18ee20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:22.648506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.649608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-8d22eece-ef39-4830-9513-72e9a577cb60 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:22.651172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1137/2260] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:22 2022] DELETE /identity/v3/projects/13d7dac7e7b74443864ce5192d18ee20 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:22.656114 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc3c1a-e6fb-4dc0-9a55-9507efda7616 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.656413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc3c1a-e6fb-4dc0-9a55-9507efda7616 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.656695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc3c1a-e6fb-4dc0-9a55-9507efda7616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.676668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-59dc3c1a-e6fb-4dc0-9a55-9507efda7616 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:22.678099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-59dc3c1a-e6fb-4dc0-9a55-9507efda7616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:22.769839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1124/2261] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:22.777370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcb218-5902-439a-879b-5361f7539fbe None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.777711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcb218-5902-439a-879b-5361f7539fbe None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.778413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebcb218-5902-439a-879b-5361f7539fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.797105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1ebcb218-5902-439a-879b-5361f7539fbe None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:22.798356 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1ebcb218-5902-439a-879b-5361f7539fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:22.807293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1138/2262] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:22.811654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc3c1a-e6fb-4dc0-9a55-9507efda7616 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.822725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc3c1a-e6fb-4dc0-9a55-9507efda7616 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.827925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bbd2f-5208-4a4d-bb46-9cfcd7acba0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.828607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bbd2f-5208-4a4d-bb46-9cfcd7acba0e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.828886 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bbd2f-5208-4a4d-bb46-9cfcd7acba0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.829170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68bbd2f-5208-4a4d-bb46-9cfcd7acba0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.829955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bbd2f-5208-4a4d-bb46-9cfcd7acba0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:22.830270 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bbd2f-5208-4a4d-bb46-9cfcd7acba0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.832656 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68bbd2f-5208-4a4d-bb46-9cfcd7acba0e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.845512 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1125/2263] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:22.850655 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcb218-5902-439a-879b-5361f7539fbe None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.860793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebcb218-5902-439a-879b-5361f7539fbe None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.866548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3d3b8-9d23-47bf-9c11-c00b3b446ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.867220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3d3b8-9d23-47bf-9c11-c00b3b446ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.867507 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3d3b8-9d23-47bf-9c11-c00b3b446ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.867786 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc3d3b8-9d23-47bf-9c11-c00b3b446ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.868496 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3d3b8-9d23-47bf-9c11-c00b3b446ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.870479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc3d3b8-9d23-47bf-9c11-c00b3b446ec5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.901500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1139/2264] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:22 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:22.902802 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bbd2f-5208-4a4d-bb46-9cfcd7acba0e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:22.916851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68bbd2f-5208-4a4d-bb46-9cfcd7acba0e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:22.983580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e936d12-a2b5-4bac-a69b-b12a78578574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:22.984414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e936d12-a2b5-4bac-a69b-b12a78578574 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:22.984727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e936d12-a2b5-4bac-a69b-b12a78578574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:22.985004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e936d12-a2b5-4bac-a69b-b12a78578574 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:22.985753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e936d12-a2b5-4bac-a69b-b12a78578574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:22.987815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e936d12-a2b5-4bac-a69b-b12a78578574 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:22.996976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:23.011035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1126/2265] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:22 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:29:23.015347 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3d3b8-9d23-47bf-9c11-c00b3b446ec5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.026714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc3d3b8-9d23-47bf-9c11-c00b3b446ec5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.031522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513b579-b710-4c53-a969-1ae4292bd1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.032260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513b579-b710-4c53-a969-1ae4292bd1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.032577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513b579-b710-4c53-a969-1ae4292bd1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.032856 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4513b579-b710-4c53-a969-1ae4292bd1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.033638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513b579-b710-4c53-a969-1ae4292bd1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.036034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4513b579-b710-4c53-a969-1ae4292bd1a2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.047673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1140/2266] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:23.057286 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e936d12-a2b5-4bac-a69b-b12a78578574 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.069194 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e936d12-a2b5-4bac-a69b-b12a78578574 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.073964 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a423052-5cfd-4b8a-8bbd-a697f31c5745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.074655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a423052-5cfd-4b8a-8bbd-a697f31c5745 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.074937 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a423052-5cfd-4b8a-8bbd-a697f31c5745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.075218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a423052-5cfd-4b8a-8bbd-a697f31c5745 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.075956 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a423052-5cfd-4b8a-8bbd-a697f31c5745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.077585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a423052-5cfd-4b8a-8bbd-a697f31c5745 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.089805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1127/2267] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:23 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:23.095315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513b579-b710-4c53-a969-1ae4292bd1a2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.106249 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4513b579-b710-4c53-a969-1ae4292bd1a2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.110992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390a197-e4a7-4a9f-a2ed-6773030f97c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.111879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390a197-e4a7-4a9f-a2ed-6773030f97c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.112164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390a197-e4a7-4a9f-a2ed-6773030f97c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.112954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4390a197-e4a7-4a9f-a2ed-6773030f97c5 admin admin] PATH_INFO: `/v3/projects/a5aab80e3a134ec588ecf32eb1f0a398/users/68c2fdb3a02c42029fd40686ffb8a24a/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.130405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390a197-e4a7-4a9f-a2ed-6773030f97c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5aab80e3a134ec588ecf32eb1f0a398, user_id=68c2fdb3a02c42029fd40686ffb8a24a, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.133124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4390a197-e4a7-4a9f-a2ed-6773030f97c5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.148078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1141/2268] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:23 2022] PUT /identity/v3/projects/a5aab80e3a134ec588ecf32eb1f0a398/users/68c2fdb3a02c42029fd40686ffb8a24a/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:23.158373 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf913a09-b81a-4a4a-ad16-6dbc168ff92f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.158901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf913a09-b81a-4a4a-ad16-6dbc168ff92f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.158901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf913a09-b81a-4a4a-ad16-6dbc168ff92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.188043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bf913a09-b81a-4a4a-ad16-6dbc168ff92f None None] MFA Rules not processed for user `68c2fdb3a02c42029fd40686ffb8a24a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:23.189284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bf913a09-b81a-4a4a-ad16-6dbc168ff92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:23.261119 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1128/2269] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:23.272669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390a197-e4a7-4a9f-a2ed-6773030f97c5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:23.274983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4390a197-e4a7-4a9f-a2ed-6773030f97c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:23.372301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4390a197-e4a7-4a9f-a2ed-6773030f97c5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:23.376839 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e77b1-b526-4a56-a397-ce20bb87ad99 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ff10db6937b4f2b889c30dd5fad8111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:23.378029 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e77b1-b526-4a56-a397-ce20bb87ad99 service glance] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:23.378319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e77b1-b526-4a56-a397-ce20bb87ad99 service glance] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:23.378789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e77b1-b526-4a56-a397-ce20bb87ad99 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:23.389603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e77b1-b526-4a56-a397-ce20bb87ad99 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:23.391054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e77b1-b526-4a56-a397-ce20bb87ad99 service glance] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:23.402502 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1142/2270] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:24.407891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134237a-c56b-4229-a0de-52b772c662bf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:24.408437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134237a-c56b-4229-a0de-52b772c662bf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:24.408437 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6134237a-c56b-4229-a0de-52b772c662bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:24.440869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6134237a-c56b-4229-a0de-52b772c662bf None None] MFA Rules not processed for user `d582e9b7a49c4ea6b9a1d6986d6f4163`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:24.442300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6134237a-c56b-4229-a0de-52b772c662bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:24.507426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1129/2271] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:24.654309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e77b1-b526-4a56-a397-ce20bb87ad99 service glance] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:24.655907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0d6e77b1-b526-4a56-a397-ce20bb87ad99 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:24.731699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e77b1-b526-4a56-a397-ce20bb87ad99 service glance] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:24.736219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:24.737166 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:24.738134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:24.738376 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] PATH_INFO: `/v3/users/5bfe77d7894b4cc897f18137fa317ddc` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:24.739724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bfe77d7894b4cc897f18137fa317ddc)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:24.741724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:24.754064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:24.821144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bfe77d7894b4cc897f18137fa317ddc'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:24.829736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bfe77d7894b4cc897f18137fa317ddc'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:24.830308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bfe77d7894b4cc897f18137fa317ddc'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:24.839484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bfe77d7894b4cc897f18137fa317ddc'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:24.856576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bfe77d7894b4cc897f18137fa317ddc'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:24.865062 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1143/2272] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:24 2022] DELETE /identity/v3/users/5bfe77d7894b4cc897f18137fa317ddc => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:24.870150 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77337fd-16ed-49d3-b01d-ffb380ef7b7b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:24.870667 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77337fd-16ed-49d3-b01d-ffb380ef7b7b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:24.871076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77337fd-16ed-49d3-b01d-ffb380ef7b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:24.891830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-d77337fd-16ed-49d3-b01d-ffb380ef7b7b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:24.893139 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d77337fd-16ed-49d3-b01d-ffb380ef7b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:24.970797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1130/2273] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:24.981221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:24.983004 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:25.063280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1a3247-69c9-4875-9078-3d134b589a15 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.067395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00888324-bdb2-467e-9f67-6d5f61df1857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.068082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00888324-bdb2-467e-9f67-6d5f61df1857 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.068364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00888324-bdb2-467e-9f67-6d5f61df1857 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.068640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00888324-bdb2-467e-9f67-6d5f61df1857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.078754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00888324-bdb2-467e-9f67-6d5f61df1857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.080222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00888324-bdb2-467e-9f67-6d5f61df1857 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.091172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1144/2274] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:25.480984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77337fd-16ed-49d3-b01d-ffb380ef7b7b None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.482751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d77337fd-16ed-49d3-b01d-ffb380ef7b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:25.495928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77337fd-16ed-49d3-b01d-ffb380ef7b7b None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.500601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.501495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.501999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.502279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] PATH_INFO: `/v3/projects/763c7470856b43ee8b139b9050fa360a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.504211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=763c7470856b43ee8b139b9050fa360a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.506327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.519523 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:25.558994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '763c7470856b43ee8b139b9050fa360a', 'request_id': 'req-72476798-7fa4-4e19-a1f8-18219256d2e0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:25.564915 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '763c7470856b43ee8b139b9050fa360a', 'request_id': 'req-72476798-7fa4-4e19-a1f8-18219256d2e0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:25.574723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '763c7470856b43ee8b139b9050fa360a', 'request_id': 'req-72476798-7fa4-4e19-a1f8-18219256d2e0'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:25.592764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] The token cache is being invalidate because project 763c7470856b43ee8b139b9050fa360a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:25.593060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:25.593733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-72476798-7fa4-4e19-a1f8-18219256d2e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:25.595314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1131/2275] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:25 2022] DELETE /identity/v3/projects/763c7470856b43ee8b139b9050fa360a => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:25.600528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86edc8-1ebc-4f9a-bab6-32f7ddba5cb2 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.600821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86edc8-1ebc-4f9a-bab6-32f7ddba5cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.601103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b86edc8-1ebc-4f9a-bab6-32f7ddba5cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.619744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4b86edc8-1ebc-4f9a-bab6-32f7ddba5cb2 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:25.621145 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4b86edc8-1ebc-4f9a-bab6-32f7ddba5cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:25.684529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1145/2276] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:25.693603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397d852-c1f9-4037-97e4-d7327777e9b5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.693897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397d852-c1f9-4037-97e4-d7327777e9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.694179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3397d852-c1f9-4037-97e4-d7327777e9b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.715572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3397d852-c1f9-4037-97e4-d7327777e9b5 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:25.716811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3397d852-c1f9-4037-97e4-d7327777e9b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:25.725352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1132/2277] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:25.729638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86edc8-1ebc-4f9a-bab6-32f7ddba5cb2 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.740521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b86edc8-1ebc-4f9a-bab6-32f7ddba5cb2 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.745258 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc077ed-055d-4b93-941f-323951a623c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.745971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc077ed-055d-4b93-941f-323951a623c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.746255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc077ed-055d-4b93-941f-323951a623c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.746520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc077ed-055d-4b93-941f-323951a623c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.747463 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc077ed-055d-4b93-941f-323951a623c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:25.747865 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc077ed-055d-4b93-941f-323951a623c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.749710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc077ed-055d-4b93-941f-323951a623c5 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.766643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1146/2278] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:25.771907 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397d852-c1f9-4037-97e4-d7327777e9b5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.781893 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3397d852-c1f9-4037-97e4-d7327777e9b5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.786879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26cc5c-a4af-4364-b203-10293b7327e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.787577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26cc5c-a4af-4364-b203-10293b7327e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.787869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26cc5c-a4af-4364-b203-10293b7327e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.788143 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af26cc5c-a4af-4364-b203-10293b7327e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.788870 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26cc5c-a4af-4364-b203-10293b7327e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.790820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af26cc5c-a4af-4364-b203-10293b7327e9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.821042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1133/2279] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:25 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:29:25.825194 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc077ed-055d-4b93-941f-323951a623c5 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.836686 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc077ed-055d-4b93-941f-323951a623c5 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.864594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:25.924387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26cc5c-a4af-4364-b203-10293b7327e9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.927296 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-af26cc5c-a4af-4364-b203-10293b7327e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:25.931973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed2806-cdd4-4c3c-922e-b14b0b301d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.932660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2806-cdd4-4c3c-922e-b14b0b301d7d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.932973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2806-cdd4-4c3c-922e-b14b0b301d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.933231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed2806-cdd4-4c3c-922e-b14b0b301d7d admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.934003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed2806-cdd4-4c3c-922e-b14b0b301d7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.936810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed2806-cdd4-4c3c-922e-b14b0b301d7d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.942460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af26cc5c-a4af-4364-b203-10293b7327e9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.947366 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.948428 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.948723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.949000 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] PATH_INFO: `/v3/users/17d17c97e4f64c898e63f0546465f6f3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.950386 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d17c97e4f64c898e63f0546465f6f3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.952515 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:25.967971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1147/2280] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:25 2022] POST /identity/v3/users => generated 294 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:29:25.969978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed2806-cdd4-4c3c-922e-b14b0b301d7d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:25.982289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed2806-cdd4-4c3c-922e-b14b0b301d7d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:25.988061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd718832-cdbe-4335-866d-5f68e928c45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:25.988927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd718832-cdbe-4335-866d-5f68e928c45b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:25.989268 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd718832-cdbe-4335-866d-5f68e928c45b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:25.989612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd718832-cdbe-4335-866d-5f68e928c45b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:25.990367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd718832-cdbe-4335-866d-5f68e928c45b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:25.992084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd718832-cdbe-4335-866d-5f68e928c45b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.010733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1148/2281] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:29:26.019736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd718832-cdbe-4335-866d-5f68e928c45b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.034955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd718832-cdbe-4335-866d-5f68e928c45b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.035148 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d17c97e4f64c898e63f0546465f6f3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:26.040467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb32ad-efc5-4b13-a379-1783180b335f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.041163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb32ad-efc5-4b13-a379-1783180b335f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.041522 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb32ad-efc5-4b13-a379-1783180b335f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.041825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feb32ad-efc5-4b13-a379-1783180b335f admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.042485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb32ad-efc5-4b13-a379-1783180b335f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.044253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feb32ad-efc5-4b13-a379-1783180b335f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.049654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d17c97e4f64c898e63f0546465f6f3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:26.051941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d17c97e4f64c898e63f0546465f6f3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:26.061445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1149/2282] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:26.065086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d17c97e4f64c898e63f0546465f6f3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:26.065266 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb32ad-efc5-4b13-a379-1783180b335f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.074559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7092f0ce-3f41-4d37-a42b-2e3a947537ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d17c97e4f64c898e63f0546465f6f3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:26.076472 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feb32ad-efc5-4b13-a379-1783180b335f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.081440 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed07cbe1-81ba-4600-884e-97884ed65f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.084682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1134/2283] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:25 2022] DELETE /identity/v3/users/17d17c97e4f64c898e63f0546465f6f3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:26.084822 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07cbe1-81ba-4600-884e-97884ed65f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.084822 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07cbe1-81ba-4600-884e-97884ed65f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.084822 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed07cbe1-81ba-4600-884e-97884ed65f28 admin admin] PATH_INFO: `/v3/projects/898a3f51f8bf47beb33091fe4694bab8/users/d76acfbe39f34168b9117007357a304e/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.087135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33304c5-d55f-416a-acd2-2c7f6500d0e0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.087135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33304c5-d55f-416a-acd2-2c7f6500d0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.087135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33304c5-d55f-416a-acd2-2c7f6500d0e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.098567 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07cbe1-81ba-4600-884e-97884ed65f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=898a3f51f8bf47beb33091fe4694bab8, user_id=d76acfbe39f34168b9117007357a304e, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.101243 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed07cbe1-81ba-4600-884e-97884ed65f28 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.103341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f33304c5-d55f-416a-acd2-2c7f6500d0e0 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:26.104557 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f33304c5-d55f-416a-acd2-2c7f6500d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:26.115491 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1150/2284] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:26 2022] PUT /identity/v3/projects/898a3f51f8bf47beb33091fe4694bab8/users/d76acfbe39f34168b9117007357a304e/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:29:26.127334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786e0a6-e762-4832-a68c-fb8a9efa6746 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.127666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786e0a6-e762-4832-a68c-fb8a9efa6746 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.128055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786e0a6-e762-4832-a68c-fb8a9efa6746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.157547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-0786e0a6-e762-4832-a68c-fb8a9efa6746 None None] MFA Rules not processed for user `d76acfbe39f34168b9117007357a304e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:26.158792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0786e0a6-e762-4832-a68c-fb8a9efa6746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:26.162195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.183608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1135/2285] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:26.194414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33304c5-d55f-416a-acd2-2c7f6500d0e0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.196002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f33304c5-d55f-416a-acd2-2c7f6500d0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:26.231163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1151/2286] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:26.235429 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786e0a6-e762-4832-a68c-fb8a9efa6746 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.236746 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-0786e0a6-e762-4832-a68c-fb8a9efa6746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:26.263871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33304c5-d55f-416a-acd2-2c7f6500d0e0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.326394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.331718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1005885-e8ce-4858-afce-cd77c5b8dec4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.332385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1005885-e8ce-4858-afce-cd77c5b8dec4 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.332672 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1005885-e8ce-4858-afce-cd77c5b8dec4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.332999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1005885-e8ce-4858-afce-cd77c5b8dec4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.355680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786e0a6-e762-4832-a68c-fb8a9efa6746 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.364344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa2e19-8d9d-47c7-b592-3b6e22378d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.365196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa2e19-8d9d-47c7-b592-3b6e22378d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.365781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa2e19-8d9d-47c7-b592-3b6e22378d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.378366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa2e19-8d9d-47c7-b592-3b6e22378d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.379187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa2e19-8d9d-47c7-b592-3b6e22378d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.381719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fa2e19-8d9d-47c7-b592-3b6e22378d24 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.413329 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1152/2287] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:26 2022] POST /identity/v3/projects => generated 385 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:29:26.417234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa2e19-8d9d-47c7-b592-3b6e22378d24 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.422551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1005885-e8ce-4858-afce-cd77c5b8dec4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.424502 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1005885-e8ce-4858-afce-cd77c5b8dec4 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.462358 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1136/2288] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:29:26.522374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa2e19-8d9d-47c7-b592-3b6e22378d24 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.527807 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccbb88-2307-4883-9886-b88d0ccd7d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.528978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccbb88-2307-4883-9886-b88d0ccd7d23 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.529262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccbb88-2307-4883-9886-b88d0ccd7d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.529735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffccbb88-2307-4883-9886-b88d0ccd7d23 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.530727 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccbb88-2307-4883-9886-b88d0ccd7d23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.532930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffccbb88-2307-4883-9886-b88d0ccd7d23 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.559584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1153/2289] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:26 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:26.564174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1005885-e8ce-4858-afce-cd77c5b8dec4 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.577491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1005885-e8ce-4858-afce-cd77c5b8dec4 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.584212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714d3d4-b986-45ce-b1e4-b3dfc96db2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.585047 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714d3d4-b986-45ce-b1e4-b3dfc96db2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.585347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714d3d4-b986-45ce-b1e4-b3dfc96db2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.585767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714d3d4-b986-45ce-b1e4-b3dfc96db2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.586650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714d3d4-b986-45ce-b1e4-b3dfc96db2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.588619 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714d3d4-b986-45ce-b1e4-b3dfc96db2c9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.603390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1137/2290] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:26.609074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccbb88-2307-4883-9886-b88d0ccd7d23 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.624470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffccbb88-2307-4883-9886-b88d0ccd7d23 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.632547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02abbec3-e047-4e59-875f-1877f5611fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.633528 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abbec3-e047-4e59-875f-1877f5611fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.633820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abbec3-e047-4e59-875f-1877f5611fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.634095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02abbec3-e047-4e59-875f-1877f5611fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.635001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abbec3-e047-4e59-875f-1877f5611fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.637866 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02abbec3-e047-4e59-875f-1877f5611fe6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.657428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1154/2291] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:26.659609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714d3d4-b986-45ce-b1e4-b3dfc96db2c9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.671415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714d3d4-b986-45ce-b1e4-b3dfc96db2c9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.678965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6759c6-1ab8-4d18-a4b5-6cb4723b4d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.680225 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6759c6-1ab8-4d18-a4b5-6cb4723b4d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.680710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6759c6-1ab8-4d18-a4b5-6cb4723b4d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.681160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6759c6-1ab8-4d18-a4b5-6cb4723b4d29 admin admin] PATH_INFO: `/v3/projects/0269caaedd604c2dad48a3477a408f75/users/7afb9ddd511d4ba8b2d576716866d77b/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.709365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6759c6-1ab8-4d18-a4b5-6cb4723b4d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0269caaedd604c2dad48a3477a408f75, user_id=7afb9ddd511d4ba8b2d576716866d77b, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.713758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6759c6-1ab8-4d18-a4b5-6cb4723b4d29 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:26.732171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1138/2292] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:26 2022] PUT /identity/v3/projects/0269caaedd604c2dad48a3477a408f75/users/7afb9ddd511d4ba8b2d576716866d77b/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:26.745530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394ee79-33db-4cb1-a2e4-305ea2634af6 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.746104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394ee79-33db-4cb1-a2e4-305ea2634af6 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.746363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a394ee79-33db-4cb1-a2e4-305ea2634af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.791143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a394ee79-33db-4cb1-a2e4-305ea2634af6 None None] MFA Rules not processed for user `7afb9ddd511d4ba8b2d576716866d77b`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:26.792418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a394ee79-33db-4cb1-a2e4-305ea2634af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:26.889112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1155/2293] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:26.894724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6759c6-1ab8-4d18-a4b5-6cb4723b4d29 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:26.924362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:26.980824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6759c6-1ab8-4d18-a4b5-6cb4723b4d29 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:26.987744 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127bd51-91c1-4604-aa3d-de5f6c4017f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:26.988629 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127bd51-91c1-4604-aa3d-de5f6c4017f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:26.989002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127bd51-91c1-4604-aa3d-de5f6c4017f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:26.989413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7127bd51-91c1-4604-aa3d-de5f6c4017f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:26.990650 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127bd51-91c1-4604-aa3d-de5f6c4017f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:26.993273 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7127bd51-91c1-4604-aa3d-de5f6c4017f6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.028133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1139/2294] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:26 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:27.032824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394ee79-33db-4cb1-a2e4-305ea2634af6 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.047675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a394ee79-33db-4cb1-a2e4-305ea2634af6 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.051638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127bd51-91c1-4604-aa3d-de5f6c4017f6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.053857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7127bd51-91c1-4604-aa3d-de5f6c4017f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:27.134374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:27.155565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0933295-80ce-44d3-a0d8-e970c5f7f2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.156885 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0933295-80ce-44d3-a0d8-e970c5f7f2f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.157179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0933295-80ce-44d3-a0d8-e970c5f7f2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.157487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0933295-80ce-44d3-a0d8-e970c5f7f2f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.158216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0933295-80ce-44d3-a0d8-e970c5f7f2f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.161038 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0933295-80ce-44d3-a0d8-e970c5f7f2f0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.170227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7127bd51-91c1-4604-aa3d-de5f6c4017f6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.176924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.177867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.178168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.178608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] PATH_INFO: `/v3/projects/4f1801764ad8413cb57728a1277c0ea5` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.179858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f1801764ad8413cb57728a1277c0ea5)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.181851 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.184722 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1156/2295] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:27 2022] POST /identity/v3/users => generated 299 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:27.189011 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0933295-80ce-44d3-a0d8-e970c5f7f2f0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.201933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0933295-80ce-44d3-a0d8-e970c5f7f2f0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.208568 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30b5d6-bab7-4dc9-80d5-a50491af5044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.209569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30b5d6-bab7-4dc9-80d5-a50491af5044 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.210253 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30b5d6-bab7-4dc9-80d5-a50491af5044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.210545 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30b5d6-bab7-4dc9-80d5-a50491af5044 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.211321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30b5d6-bab7-4dc9-80d5-a50491af5044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.213837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a30b5d6-bab7-4dc9-80d5-a50491af5044 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.242466 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1157/2296] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:29:27.251481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30b5d6-bab7-4dc9-80d5-a50491af5044 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.256684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f1801764ad8413cb57728a1277c0ea5', 'request_id': 'req-0cbe5466-38b7-4568-bc99-30f1ae473354'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:27.265495 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a30b5d6-bab7-4dc9-80d5-a50491af5044 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.265750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f1801764ad8413cb57728a1277c0ea5', 'request_id': 'req-0cbe5466-38b7-4568-bc99-30f1ae473354'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:27.277914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f1801764ad8413cb57728a1277c0ea5', 'request_id': 'req-0cbe5466-38b7-4568-bc99-30f1ae473354'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:27.291215 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] The token cache is being invalidate because project 4f1801764ad8413cb57728a1277c0ea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:27.291484 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:27.292178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:27.293816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1140/2297] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:27 2022] DELETE /identity/v3/projects/4f1801764ad8413cb57728a1277c0ea5 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:27.360366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5613f-c7bc-48e6-b9fe-7517aedd2eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.361446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5613f-c7bc-48e6-b9fe-7517aedd2eca admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.361813 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5613f-c7bc-48e6-b9fe-7517aedd2eca admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.362169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5613f-c7bc-48e6-b9fe-7517aedd2eca admin admin] PATH_INFO: `/v3/projects/413639ea2f75459eb79245cfd5ccfc2f/users/9fc19551d26b40e09bc3f7cef893e16d/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.400277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5613f-c7bc-48e6-b9fe-7517aedd2eca admin admin] RBAC: Authorizing `identity:create_grant(project_id=413639ea2f75459eb79245cfd5ccfc2f, user_id=9fc19551d26b40e09bc3f7cef893e16d, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.404443 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5613f-c7bc-48e6-b9fe-7517aedd2eca admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.427295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1158/2298] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:27 2022] PUT /identity/v3/projects/413639ea2f75459eb79245cfd5ccfc2f/users/9fc19551d26b40e09bc3f7cef893e16d/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:29:27.432443 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.442159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:27.468138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbe5466-38b7-4568-bc99-30f1ae473354 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.514116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:27.547747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f9f92-31ca-402b-a830-b67d4f02b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.549104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f9f92-31ca-402b-a830-b67d4f02b54d admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.549449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f9f92-31ca-402b-a830-b67d4f02b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.550351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f9f92-31ca-402b-a830-b67d4f02b54d admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.555009 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f9f92-31ca-402b-a830-b67d4f02b54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.557163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93f9f92-31ca-402b-a830-b67d4f02b54d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.575972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1141/2299] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:27.580269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5613f-c7bc-48e6-b9fe-7517aedd2eca admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.594270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5613f-c7bc-48e6-b9fe-7517aedd2eca admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.599388 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda6e8-e203-478e-b633-5b6c5ef2bc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.600304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda6e8-e203-478e-b633-5b6c5ef2bc0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.600592 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda6e8-e203-478e-b633-5b6c5ef2bc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.600960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cda6e8-e203-478e-b633-5b6c5ef2bc0b admin admin] PATH_INFO: `/v3/projects/413639ea2f75459eb79245cfd5ccfc2f/users/9fc19551d26b40e09bc3f7cef893e16d/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.603076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda6e8-e203-478e-b633-5b6c5ef2bc0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=413639ea2f75459eb79245cfd5ccfc2f, user_id=9fc19551d26b40e09bc3f7cef893e16d, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.605908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cda6e8-e203-478e-b633-5b6c5ef2bc0b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.618283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1159/2300] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:27 2022] PUT /identity/v3/projects/413639ea2f75459eb79245cfd5ccfc2f/users/9fc19551d26b40e09bc3f7cef893e16d/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:27.630780 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d39dd3-ddde-4236-b2b8-09089157696f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.631264 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d39dd3-ddde-4236-b2b8-09089157696f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.631593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d39dd3-ddde-4236-b2b8-09089157696f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.662743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-57d39dd3-ddde-4236-b2b8-09089157696f None None] MFA Rules not processed for user `9fc19551d26b40e09bc3f7cef893e16d`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:27.664049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-57d39dd3-ddde-4236-b2b8-09089157696f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:27.770249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1142/2301] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:27.782712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda6e8-e203-478e-b633-5b6c5ef2bc0b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:27.785366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-70cda6e8-e203-478e-b633-5b6c5ef2bc0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:27.851996 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cda6e8-e203-478e-b633-5b6c5ef2bc0b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:27.856753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0600a4-3d63-46b4-9b20-f25fcdfc9895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:27.857594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0600a4-3d63-46b4-9b20-f25fcdfc9895 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:27.858712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0600a4-3d63-46b4-9b20-f25fcdfc9895 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:27.859556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0600a4-3d63-46b4-9b20-f25fcdfc9895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:27.872321 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0600a4-3d63-46b4-9b20-f25fcdfc9895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:27.873969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0600a4-3d63-46b4-9b20-f25fcdfc9895 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:27.890358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1160/2302] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:27 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:28.582423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d39dd3-ddde-4236-b2b8-09089157696f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:28.593037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d39dd3-ddde-4236-b2b8-09089157696f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:28.597790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf69760-57c7-465c-82a8-c048dc0987bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:28.598479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf69760-57c7-465c-82a8-c048dc0987bd service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:28.598776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf69760-57c7-465c-82a8-c048dc0987bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:28.599048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf69760-57c7-465c-82a8-c048dc0987bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:28.601458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ccf69760-57c7-465c-82a8-c048dc0987bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:28.676371 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf69760-57c7-465c-82a8-c048dc0987bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:28.678015 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf69760-57c7-465c-82a8-c048dc0987bd service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:28.690019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1143/2303] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:28 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:33.954745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0600a4-3d63-46b4-9b20-f25fcdfc9895 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:33.956238 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8d0600a4-3d63-46b4-9b20-f25fcdfc9895 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:33.968281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0600a4-3d63-46b4-9b20-f25fcdfc9895 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:33.971967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a39f45-1710-4aca-8e50-f20f36e916a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:33.972609 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a39f45-1710-4aca-8e50-f20f36e916a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:33.972894 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a39f45-1710-4aca-8e50-f20f36e916a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:33.973177 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a39f45-1710-4aca-8e50-f20f36e916a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:33.976941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a0a39f45-1710-4aca-8e50-f20f36e916a5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:34.032160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:34.062779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a39f45-1710-4aca-8e50-f20f36e916a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:34.064628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a39f45-1710-4aca-8e50-f20f36e916a5 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:34.075879 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1161/2304] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:36.368968 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf69760-57c7-465c-82a8-c048dc0987bd service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:36.435228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:36.452269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf69760-57c7-465c-82a8-c048dc0987bd service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:36.456881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:36.457814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.458101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.458406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] PATH_INFO: `/v3/users/d76acfbe39f34168b9117007357a304e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.460948 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d76acfbe39f34168b9117007357a304e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:36.462939 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:36.516533 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd76acfbe39f34168b9117007357a304e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:36.525005 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd76acfbe39f34168b9117007357a304e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:36.525776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd76acfbe39f34168b9117007357a304e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:36.543409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd76acfbe39f34168b9117007357a304e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:36.549410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd76acfbe39f34168b9117007357a304e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:36.558577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1144/2305] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:36 2022] DELETE /identity/v3/users/d76acfbe39f34168b9117007357a304e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:36.563995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e59ad-e4fa-467a-b9e8-153c88278062 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.564619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e59ad-e4fa-467a-b9e8-153c88278062 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.564936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e59ad-e4fa-467a-b9e8-153c88278062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.588335 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-639e59ad-e4fa-467a-b9e8-153c88278062 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:36.589717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-639e59ad-e4fa-467a-b9e8-153c88278062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:36.667898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1162/2306] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:36.678836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:36.680179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:36.743295 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2f77c-7f25-4f2a-9c1b-5205f2f886d8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:36.747128 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15611f9a-5666-4ec8-bd8a-4d662adb6238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:36.747784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15611f9a-5666-4ec8-bd8a-4d662adb6238 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:36.748066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15611f9a-5666-4ec8-bd8a-4d662adb6238 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:36.748369 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15611f9a-5666-4ec8-bd8a-4d662adb6238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:36.759312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15611f9a-5666-4ec8-bd8a-4d662adb6238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:36.760808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15611f9a-5666-4ec8-bd8a-4d662adb6238 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:36.770840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1145/2307] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:37.159204 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e59ad-e4fa-467a-b9e8-153c88278062 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.161578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-639e59ad-e4fa-467a-b9e8-153c88278062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:37.179938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e59ad-e4fa-467a-b9e8-153c88278062 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.185303 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.186795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.187112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.187532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] PATH_INFO: `/v3/projects/898a3f51f8bf47beb33091fe4694bab8` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.189042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=898a3f51f8bf47beb33091fe4694bab8)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:37.191102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:37.243207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '898a3f51f8bf47beb33091fe4694bab8', 'request_id': 'req-dbd0baae-39cd-46ad-a26a-4381d6daec73'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.250979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '898a3f51f8bf47beb33091fe4694bab8', 'request_id': 'req-dbd0baae-39cd-46ad-a26a-4381d6daec73'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.259753 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '898a3f51f8bf47beb33091fe4694bab8', 'request_id': 'req-dbd0baae-39cd-46ad-a26a-4381d6daec73'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.272955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] The token cache is being invalidate because project 898a3f51f8bf47beb33091fe4694bab8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:37.273363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.274099 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.275740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1163/2308] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:37 2022] DELETE /identity/v3/projects/898a3f51f8bf47beb33091fe4694bab8 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:37.279853 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15611f9a-5666-4ec8-bd8a-4d662adb6238 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.294913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15611f9a-5666-4ec8-bd8a-4d662adb6238 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.382511 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.384191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.384710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.385230 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] PATH_INFO: `/v3/users/7afb9ddd511d4ba8b2d576716866d77b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.387490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] RBAC: Authorizing `identity:delete_user(user_id=7afb9ddd511d4ba8b2d576716866d77b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:37.391344 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:37.413571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:37.461716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7afb9ddd511d4ba8b2d576716866d77b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.475937 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7afb9ddd511d4ba8b2d576716866d77b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.476957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7afb9ddd511d4ba8b2d576716866d77b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.492111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7afb9ddd511d4ba8b2d576716866d77b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.501718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7afb9ddd511d4ba8b2d576716866d77b'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:37.512277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1146/2309] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:37 2022] DELETE /identity/v3/users/7afb9ddd511d4ba8b2d576716866d77b => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:29:37.824980 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:37.827936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:37.852749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd0baae-39cd-46ad-a26a-4381d6daec73 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:37.974088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:37.991833 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:37.994113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:37.994892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:37.995645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] PATH_INFO: `/v3/projects/0269caaedd604c2dad48a3477a408f75` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:37.998128 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0269caaedd604c2dad48a3477a408f75)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.002993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.061825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0269caaedd604c2dad48a3477a408f75', 'request_id': 'req-6db83896-bdf5-447c-89c3-3c1ac62e6c13'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.070366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0269caaedd604c2dad48a3477a408f75', 'request_id': 'req-6db83896-bdf5-447c-89c3-3c1ac62e6c13'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.078901 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0269caaedd604c2dad48a3477a408f75', 'request_id': 'req-6db83896-bdf5-447c-89c3-3c1ac62e6c13'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.090911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] The token cache is being invalidate because project 0269caaedd604c2dad48a3477a408f75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:38.091198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.091825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.093362 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1164/2310] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:37 2022] DELETE /identity/v3/projects/0269caaedd604c2dad48a3477a408f75 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:38.097424 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.098834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:38.112143 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d8bbd-ab41-4a48-9edc-8bebc8502aee admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.170582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.171516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.171803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.172092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] PATH_INFO: `/v3/users/9fc19551d26b40e09bc3f7cef893e16d` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.173355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fc19551d26b40e09bc3f7cef893e16d)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.175367 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.226243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fc19551d26b40e09bc3f7cef893e16d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.234057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fc19551d26b40e09bc3f7cef893e16d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.234778 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fc19551d26b40e09bc3f7cef893e16d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.253423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fc19551d26b40e09bc3f7cef893e16d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.262710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-a6f3a32b-651c-4ef6-90b8-b53294e8b054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fc19551d26b40e09bc3f7cef893e16d'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.270927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1147/2311] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:38 2022] DELETE /identity/v3/users/9fc19551d26b40e09bc3f7cef893e16d => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:38.572172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.573872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:38.587336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db83896-bdf5-447c-89c3-3c1ac62e6c13 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.650602 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.651548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.651900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.652210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] PATH_INFO: `/v3/projects/413639ea2f75459eb79245cfd5ccfc2f` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.654085 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] RBAC: Authorizing `identity:delete_project(project_id=413639ea2f75459eb79245cfd5ccfc2f)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.656117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.712106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '413639ea2f75459eb79245cfd5ccfc2f', 'request_id': 'req-34cd7d01-b3be-4a63-831d-8c36d113c28e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.719454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '413639ea2f75459eb79245cfd5ccfc2f', 'request_id': 'req-34cd7d01-b3be-4a63-831d-8c36d113c28e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.729055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '413639ea2f75459eb79245cfd5ccfc2f', 'request_id': 'req-34cd7d01-b3be-4a63-831d-8c36d113c28e'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.746052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] The token cache is being invalidate because project 413639ea2f75459eb79245cfd5ccfc2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:38.746349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.747163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-34cd7d01-b3be-4a63-831d-8c36d113c28e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:38.753144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1165/2312] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:38 2022] DELETE /identity/v3/projects/413639ea2f75459eb79245cfd5ccfc2f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:38.755596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0298a1-fbd4-4267-bdab-f3e71882b3dc None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.756599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0298a1-fbd4-4267-bdab-f3e71882b3dc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.756952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0298a1-fbd4-4267-bdab-f3e71882b3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.780934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ce0298a1-fbd4-4267-bdab-f3e71882b3dc None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:38.782371 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ce0298a1-fbd4-4267-bdab-f3e71882b3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:38.789647 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:38.867144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1148/2313] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:38.875653 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed0d0b-67ed-422f-85af-4a9123a9c015 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.876203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed0d0b-67ed-422f-85af-4a9123a9c015 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.876308 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ed0d0b-67ed-422f-85af-4a9123a9c015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.895187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-a9ed0d0b-67ed-422f-85af-4a9123a9c015 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:38.897172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a9ed0d0b-67ed-422f-85af-4a9123a9c015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:38.904961 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1166/2314] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:38.909171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0298a1-fbd4-4267-bdab-f3e71882b3dc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.919973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0298a1-fbd4-4267-bdab-f3e71882b3dc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.924625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ca2f7-bb19-4a5a-860d-581caad0a26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.925285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ca2f7-bb19-4a5a-860d-581caad0a26e admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.925758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ca2f7-bb19-4a5a-860d-581caad0a26e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.926043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71ca2f7-bb19-4a5a-860d-581caad0a26e admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.926784 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ca2f7-bb19-4a5a-860d-581caad0a26e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:38.927097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ca2f7-bb19-4a5a-860d-581caad0a26e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.928824 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71ca2f7-bb19-4a5a-860d-581caad0a26e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.941292 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1149/2315] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:38.945817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed0d0b-67ed-422f-85af-4a9123a9c015 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:38.955113 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ed0d0b-67ed-422f-85af-4a9123a9c015 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:38.960817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac68f83-5ba4-4741-a86c-98ee7be8937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:38.961503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac68f83-5ba4-4741-a86c-98ee7be8937d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:38.961795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac68f83-5ba4-4741-a86c-98ee7be8937d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:38.962075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac68f83-5ba4-4741-a86c-98ee7be8937d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:38.962776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac68f83-5ba4-4741-a86c-98ee7be8937d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:38.964732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac68f83-5ba4-4741-a86c-98ee7be8937d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:38.973314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:38.995295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1167/2316] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:38 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:38.999480 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ca2f7-bb19-4a5a-860d-581caad0a26e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.010191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71ca2f7-bb19-4a5a-860d-581caad0a26e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.082970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63417f32-4789-4b96-a064-bd90f471e2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.083682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63417f32-4789-4b96-a064-bd90f471e2e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.084558 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63417f32-4789-4b96-a064-bd90f471e2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.084847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63417f32-4789-4b96-a064-bd90f471e2e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.085590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63417f32-4789-4b96-a064-bd90f471e2e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.087566 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63417f32-4789-4b96-a064-bd90f471e2e5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.108138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1150/2317] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:38 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:39.112218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac68f83-5ba4-4741-a86c-98ee7be8937d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.124114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac68f83-5ba4-4741-a86c-98ee7be8937d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.128715 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2baeec-8504-4c15-9699-c9f0de1ba3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.129403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2baeec-8504-4c15-9699-c9f0de1ba3f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.129687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2baeec-8504-4c15-9699-c9f0de1ba3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.129965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2baeec-8504-4c15-9699-c9f0de1ba3f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.131324 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2baeec-8504-4c15-9699-c9f0de1ba3f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.132995 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2baeec-8504-4c15-9699-c9f0de1ba3f4 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.144042 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1168/2318] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:39.148409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63417f32-4789-4b96-a064-bd90f471e2e5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.158771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63417f32-4789-4b96-a064-bd90f471e2e5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.164982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de80b9e-05d1-4a10-bbfe-c57eaf1025dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.165913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de80b9e-05d1-4a10-bbfe-c57eaf1025dd admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.166253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de80b9e-05d1-4a10-bbfe-c57eaf1025dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.166595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de80b9e-05d1-4a10-bbfe-c57eaf1025dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.167341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de80b9e-05d1-4a10-bbfe-c57eaf1025dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.169728 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de80b9e-05d1-4a10-bbfe-c57eaf1025dd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.181221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1151/2319] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:29:39.185330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2baeec-8504-4c15-9699-c9f0de1ba3f4 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.196667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2baeec-8504-4c15-9699-c9f0de1ba3f4 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.201164 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e519b6-f692-4413-9c86-bf6ddbd52da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.202089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e519b6-f692-4413-9c86-bf6ddbd52da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.202373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e519b6-f692-4413-9c86-bf6ddbd52da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.202645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e519b6-f692-4413-9c86-bf6ddbd52da9 admin admin] PATH_INFO: `/v3/projects/860748118a00426990feabd75f3d8505/users/b09cdb2c122d48d3a4433ed1d5daa6e3/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.218137 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e519b6-f692-4413-9c86-bf6ddbd52da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=860748118a00426990feabd75f3d8505, user_id=b09cdb2c122d48d3a4433ed1d5daa6e3, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.220782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e519b6-f692-4413-9c86-bf6ddbd52da9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.234808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1169/2320] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:39 2022] PUT /identity/v3/projects/860748118a00426990feabd75f3d8505/users/b09cdb2c122d48d3a4433ed1d5daa6e3/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:39.246347 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eedd9-3b3e-4cd9-84ce-dac16f0cb75f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.246645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eedd9-3b3e-4cd9-84ce-dac16f0cb75f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.246929 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076eedd9-3b3e-4cd9-84ce-dac16f0cb75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.277340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-076eedd9-3b3e-4cd9-84ce-dac16f0cb75f None None] MFA Rules not processed for user `b09cdb2c122d48d3a4433ed1d5daa6e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:39.278648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-076eedd9-3b3e-4cd9-84ce-dac16f0cb75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:39.364741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1152/2321] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:39.378400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e519b6-f692-4413-9c86-bf6ddbd52da9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:39.380200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-16e519b6-f692-4413-9c86-bf6ddbd52da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:39.446644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e519b6-f692-4413-9c86-bf6ddbd52da9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:39.450848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830467f-07ab-4307-ab7c-eb36ddd00c2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:39.451640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830467f-07ab-4307-ab7c-eb36ddd00c2a service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:39.451946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830467f-07ab-4307-ab7c-eb36ddd00c2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:39.452261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830467f-07ab-4307-ab7c-eb36ddd00c2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:39.462739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830467f-07ab-4307-ab7c-eb36ddd00c2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:39.464218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830467f-07ab-4307-ab7c-eb36ddd00c2a service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:39.474149 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1170/2322] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:39 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:43.596579 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eedd9-3b3e-4cd9-84ce-dac16f0cb75f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.605598 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:43.662088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076eedd9-3b3e-4cd9-84ce-dac16f0cb75f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.667423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.668309 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.668596 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.668877 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] PATH_INFO: `/v3/users/b09cdb2c122d48d3a4433ed1d5daa6e3` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.670097 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b09cdb2c122d48d3a4433ed1d5daa6e3)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.672018 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.720962 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b09cdb2c122d48d3a4433ed1d5daa6e3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:43.729582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b09cdb2c122d48d3a4433ed1d5daa6e3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:43.730398 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b09cdb2c122d48d3a4433ed1d5daa6e3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:43.746212 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b09cdb2c122d48d3a4433ed1d5daa6e3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:43.754357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b09cdb2c122d48d3a4433ed1d5daa6e3'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:43.760384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1153/2323] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:43 2022] DELETE /identity/v3/users/b09cdb2c122d48d3a4433ed1d5daa6e3 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:43.764936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ebf37-4647-4710-822f-a18352c52225 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.765416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ebf37-4647-4710-822f-a18352c52225 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.765537 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ebf37-4647-4710-822f-a18352c52225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.784372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-247ebf37-4647-4710-822f-a18352c52225 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:43.786005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-247ebf37-4647-4710-822f-a18352c52225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:43.852319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1171/2324] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:43.864092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:43.865567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:43.934563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9889c255-dd58-40a7-a569-a5236a1f7b86 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:43.938320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d399b-4d91-4025-a756-0e7153907a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:43.938973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d399b-4d91-4025-a756-0e7153907a45 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:43.939254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d399b-4d91-4025-a756-0e7153907a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:43.939529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d399b-4d91-4025-a756-0e7153907a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:43.949976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d399b-4d91-4025-a756-0e7153907a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:43.951374 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d399b-4d91-4025-a756-0e7153907a45 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:43.961205 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1154/2325] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:44.257863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ebf37-4647-4710-822f-a18352c52225 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.259424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-247ebf37-4647-4710-822f-a18352c52225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:44.273160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ebf37-4647-4710-822f-a18352c52225 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.278518 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.279410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.280589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.281239 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] PATH_INFO: `/v3/projects/860748118a00426990feabd75f3d8505` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.282555 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=860748118a00426990feabd75f3d8505)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.284491 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.337757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '860748118a00426990feabd75f3d8505', 'request_id': 'req-913f81c6-0345-4d9e-9ecf-758f61eea3fc'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:44.339791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:44.344418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '860748118a00426990feabd75f3d8505', 'request_id': 'req-913f81c6-0345-4d9e-9ecf-758f61eea3fc'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:44.352963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '860748118a00426990feabd75f3d8505', 'request_id': 'req-913f81c6-0345-4d9e-9ecf-758f61eea3fc'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:44.363619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] The token cache is being invalidate because project 860748118a00426990feabd75f3d8505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:44.363914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:44.364547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-913f81c6-0345-4d9e-9ecf-758f61eea3fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:44.366695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1172/2326] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:44 2022] DELETE /identity/v3/projects/860748118a00426990feabd75f3d8505 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:44.372518 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b8a4-a334-4ace-8810-6a1e92194c06 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.372747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b8a4-a334-4ace-8810-6a1e92194c06 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.373052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed8b8a4-a334-4ace-8810-6a1e92194c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.391456 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-bed8b8a4-a334-4ace-8810-6a1e92194c06 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:44.392696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-bed8b8a4-a334-4ace-8810-6a1e92194c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:44.419222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:44.456834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1155/2327] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:44.467660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb5690-c05e-453f-a087-29b81afab40c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.468061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb5690-c05e-453f-a087-29b81afab40c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.468350 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb5690-c05e-453f-a087-29b81afab40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.488469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f5fb5690-c05e-453f-a087-29b81afab40c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:44.489987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f5fb5690-c05e-453f-a087-29b81afab40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:44.498088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1173/2328] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:44.502516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b8a4-a334-4ace-8810-6a1e92194c06 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.513535 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed8b8a4-a334-4ace-8810-6a1e92194c06 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.518103 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888512-6715-4bc0-83ab-d7ae39cb736d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.518758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888512-6715-4bc0-83ab-d7ae39cb736d admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.519043 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888512-6715-4bc0-83ab-d7ae39cb736d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.519317 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39888512-6715-4bc0-83ab-d7ae39cb736d admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.520052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39888512-6715-4bc0-83ab-d7ae39cb736d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:44.520361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39888512-6715-4bc0-83ab-d7ae39cb736d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.522009 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39888512-6715-4bc0-83ab-d7ae39cb736d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.535390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1156/2329] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:44.540352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb5690-c05e-453f-a087-29b81afab40c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.549887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb5690-c05e-453f-a087-29b81afab40c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.555409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eb2e2-7037-4dd6-817e-1979217a7ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.556079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eb2e2-7037-4dd6-817e-1979217a7ed0 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.556511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eb2e2-7037-4dd6-817e-1979217a7ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.556804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0eb2e2-7037-4dd6-817e-1979217a7ed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.557550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eb2e2-7037-4dd6-817e-1979217a7ed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.559484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0eb2e2-7037-4dd6-817e-1979217a7ed0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.587076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1174/2330] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:44 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:44.591100 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888512-6715-4bc0-83ab-d7ae39cb736d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.601864 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39888512-6715-4bc0-83ab-d7ae39cb736d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.666241 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d0045-81ab-47a2-8a92-5ff0db143855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.666826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d0045-81ab-47a2-8a92-5ff0db143855 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.667108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d0045-81ab-47a2-8a92-5ff0db143855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.667382 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017d0045-81ab-47a2-8a92-5ff0db143855 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.668195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d0045-81ab-47a2-8a92-5ff0db143855 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.670138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017d0045-81ab-47a2-8a92-5ff0db143855 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.697430 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1157/2331] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:44 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:44.701544 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eb2e2-7037-4dd6-817e-1979217a7ed0 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.712265 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0eb2e2-7037-4dd6-817e-1979217a7ed0 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.716928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd30009-5089-4be9-9fde-c0ad84c91106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.717632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd30009-5089-4be9-9fde-c0ad84c91106 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.717921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd30009-5089-4be9-9fde-c0ad84c91106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.718201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd30009-5089-4be9-9fde-c0ad84c91106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.718937 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd30009-5089-4be9-9fde-c0ad84c91106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.720544 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd30009-5089-4be9-9fde-c0ad84c91106 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.731768 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1175/2332] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:44.735845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d0045-81ab-47a2-8a92-5ff0db143855 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.747259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017d0045-81ab-47a2-8a92-5ff0db143855 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.751951 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c85641-eb8a-4ee3-9355-deeb3e8d22b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.752620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c85641-eb8a-4ee3-9355-deeb3e8d22b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.752910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c85641-eb8a-4ee3-9355-deeb3e8d22b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.753192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c85641-eb8a-4ee3-9355-deeb3e8d22b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.753986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c85641-eb8a-4ee3-9355-deeb3e8d22b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.755597 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c85641-eb8a-4ee3-9355-deeb3e8d22b1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.766878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1158/2333] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:44.771024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd30009-5089-4be9-9fde-c0ad84c91106 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.781317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd30009-5089-4be9-9fde-c0ad84c91106 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:44.786192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed884d-8ba7-41bf-91c7-d4491c3acfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:44.787129 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed884d-8ba7-41bf-91c7-d4491c3acfdb admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.787386 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed884d-8ba7-41bf-91c7-d4491c3acfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.787669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed884d-8ba7-41bf-91c7-d4491c3acfdb admin admin] PATH_INFO: `/v3/projects/8811435beeba4e3b8e8f02e05e8b2f92/users/fe29888be0e140ada2287a5554fe10d6/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.806914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed884d-8ba7-41bf-91c7-d4491c3acfdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8811435beeba4e3b8e8f02e05e8b2f92, user_id=fe29888be0e140ada2287a5554fe10d6, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:44.809626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed884d-8ba7-41bf-91c7-d4491c3acfdb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:44.822861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1176/2334] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:44 2022] PUT /identity/v3/projects/8811435beeba4e3b8e8f02e05e8b2f92/users/fe29888be0e140ada2287a5554fe10d6/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:44.833041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2e135-16c9-4bd6-b52f-707ced726636 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:44.833335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2e135-16c9-4bd6-b52f-707ced726636 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:44.833651 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b2e135-16c9-4bd6-b52f-707ced726636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:44.862435 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f3b2e135-16c9-4bd6-b52f-707ced726636 None None] MFA Rules not processed for user `fe29888be0e140ada2287a5554fe10d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:44.863705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f3b2e135-16c9-4bd6-b52f-707ced726636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:44.930529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1159/2335] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:44.942138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed884d-8ba7-41bf-91c7-d4491c3acfdb admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:44.943863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-44ed884d-8ba7-41bf-91c7-d4491c3acfdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:45.006208 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed884d-8ba7-41bf-91c7-d4491c3acfdb admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:45.010775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae6ccc-313a-4142-8b76-e6539090bb58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:45.011426 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae6ccc-313a-4142-8b76-e6539090bb58 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:45.011710 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae6ccc-313a-4142-8b76-e6539090bb58 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:45.011988 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae6ccc-313a-4142-8b76-e6539090bb58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:45.021025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae6ccc-313a-4142-8b76-e6539090bb58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:45.022548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae6ccc-313a-4142-8b76-e6539090bb58 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:45.032071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1177/2336] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:48.661192 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2e135-16c9-4bd6-b52f-707ced726636 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:48.687867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:48.725055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b2e135-16c9-4bd6-b52f-707ced726636 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:48.729775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:48.730653 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:48.730938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:48.731213 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] PATH_INFO: `/v3/users/fe29888be0e140ada2287a5554fe10d6` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:48.732408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe29888be0e140ada2287a5554fe10d6)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:48.734402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:48.786101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe29888be0e140ada2287a5554fe10d6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:48.793621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe29888be0e140ada2287a5554fe10d6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:48.794310 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe29888be0e140ada2287a5554fe10d6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:48.806066 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe29888be0e140ada2287a5554fe10d6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:48.814631 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe29888be0e140ada2287a5554fe10d6'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:48.820693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1160/2337] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:48 2022] DELETE /identity/v3/users/fe29888be0e140ada2287a5554fe10d6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:48.825144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa78b-3e46-4e4c-8aef-efde470e5c0f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:48.825665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa78b-3e46-4e4c-8aef-efde470e5c0f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:48.825774 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018aa78b-3e46-4e4c-8aef-efde470e5c0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:48.843437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-018aa78b-3e46-4e4c-8aef-efde470e5c0f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:48.844650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-018aa78b-3e46-4e4c-8aef-efde470e5c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:48.912207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1178/2338] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:48.923002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:48.924397 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:48.982836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044c02a0-ed7b-446b-9ff8-005a6c01820a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:48.986717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e29cb3-b204-426b-a2fe-fadb2c99bf91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:48.987405 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e29cb3-b204-426b-a2fe-fadb2c99bf91 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:48.987696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e29cb3-b204-426b-a2fe-fadb2c99bf91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:48.987961 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e29cb3-b204-426b-a2fe-fadb2c99bf91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:48.997757 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e29cb3-b204-426b-a2fe-fadb2c99bf91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:48.999190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e29cb3-b204-426b-a2fe-fadb2c99bf91 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.010648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1161/2339] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:49.297230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aa78b-3e46-4e4c-8aef-efde470e5c0f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.298739 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-018aa78b-3e46-4e4c-8aef-efde470e5c0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:49.332315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018aa78b-3e46-4e4c-8aef-efde470e5c0f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.341222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.342052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.342349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.343235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] PATH_INFO: `/v3/projects/8811435beeba4e3b8e8f02e05e8b2f92` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.344487 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8811435beeba4e3b8e8f02e05e8b2f92)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.346484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.372111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:49.408425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8811435beeba4e3b8e8f02e05e8b2f92', 'request_id': 'req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:49.416288 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8811435beeba4e3b8e8f02e05e8b2f92', 'request_id': 'req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:49.424268 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8811435beeba4e3b8e8f02e05e8b2f92', 'request_id': 'req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:49.440524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] The token cache is being invalidate because project 8811435beeba4e3b8e8f02e05e8b2f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:49.440817 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:49.441521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9ad7b515-2ed9-46c7-b19c-9c09a0deebeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:49.443447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1179/2340] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:49 2022] DELETE /identity/v3/projects/8811435beeba4e3b8e8f02e05e8b2f92 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:49.448618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026f920-dc27-4e42-b053-ebd4c069942f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.448977 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026f920-dc27-4e42-b053-ebd4c069942f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.449332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c026f920-dc27-4e42-b053-ebd4c069942f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.468953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c026f920-dc27-4e42-b053-ebd4c069942f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:49.470421 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c026f920-dc27-4e42-b053-ebd4c069942f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:49.521686 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:49.545525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1162/2341] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:49.549057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4632251-913e-4f97-bda2-3b1fff771a78 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.549450 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4632251-913e-4f97-bda2-3b1fff771a78 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.549803 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4632251-913e-4f97-bda2-3b1fff771a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.572174 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b4632251-913e-4f97-bda2-3b1fff771a78 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:49.573851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b4632251-913e-4f97-bda2-3b1fff771a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:49.586557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1180/2342] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:49.590689 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026f920-dc27-4e42-b053-ebd4c069942f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.601992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c026f920-dc27-4e42-b053-ebd4c069942f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.611090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92f3a4-afb6-4957-9579-fd4637aabd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.611817 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92f3a4-afb6-4957-9579-fd4637aabd09 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.612254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92f3a4-afb6-4957-9579-fd4637aabd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.612537 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92f3a4-afb6-4957-9579-fd4637aabd09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.613277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92f3a4-afb6-4957-9579-fd4637aabd09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:49.613621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92f3a4-afb6-4957-9579-fd4637aabd09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.615303 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92f3a4-afb6-4957-9579-fd4637aabd09 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.627609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1163/2343] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:29:49.632873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4632251-913e-4f97-bda2-3b1fff771a78 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.643438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4632251-913e-4f97-bda2-3b1fff771a78 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.649801 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37578f81-c7a8-4044-899f-e8f6a483805a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.650506 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37578f81-c7a8-4044-899f-e8f6a483805a admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.650804 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37578f81-c7a8-4044-899f-e8f6a483805a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.651068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37578f81-c7a8-4044-899f-e8f6a483805a admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.651806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37578f81-c7a8-4044-899f-e8f6a483805a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.653876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37578f81-c7a8-4044-899f-e8f6a483805a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.703564 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1181/2344] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:49 2022] POST /identity/v3/projects => generated 369 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:49.707872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92f3a4-afb6-4957-9579-fd4637aabd09 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.721338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92f3a4-afb6-4957-9579-fd4637aabd09 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.791799 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3060281-7128-4a75-8c56-78113ca9cb49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.792648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3060281-7128-4a75-8c56-78113ca9cb49 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.792876 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3060281-7128-4a75-8c56-78113ca9cb49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.793156 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3060281-7128-4a75-8c56-78113ca9cb49 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.793936 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3060281-7128-4a75-8c56-78113ca9cb49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.795901 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3060281-7128-4a75-8c56-78113ca9cb49 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.819945 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1164/2345] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:49 2022] POST /identity/v3/users => generated 285 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:49.821433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37578f81-c7a8-4044-899f-e8f6a483805a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.831738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37578f81-c7a8-4044-899f-e8f6a483805a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.837297 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc042c0-2c10-4817-bcb2-d1cf60b12c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.838049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc042c0-2c10-4817-bcb2-d1cf60b12c68 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.838336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc042c0-2c10-4817-bcb2-d1cf60b12c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.838616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc042c0-2c10-4817-bcb2-d1cf60b12c68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.839468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc042c0-2c10-4817-bcb2-d1cf60b12c68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.840951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc042c0-2c10-4817-bcb2-d1cf60b12c68 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.853414 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1182/2346] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:49.855854 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3060281-7128-4a75-8c56-78113ca9cb49 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.866175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3060281-7128-4a75-8c56-78113ca9cb49 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.870836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea703d8-a35b-4f5f-80ce-c0e388cd0579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.871509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea703d8-a35b-4f5f-80ce-c0e388cd0579 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.871791 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea703d8-a35b-4f5f-80ce-c0e388cd0579 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.872069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea703d8-a35b-4f5f-80ce-c0e388cd0579 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.872807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea703d8-a35b-4f5f-80ce-c0e388cd0579 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.874445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea703d8-a35b-4f5f-80ce-c0e388cd0579 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.888060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1165/2347] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:49.892867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc042c0-2c10-4817-bcb2-d1cf60b12c68 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:49.904250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc042c0-2c10-4817-bcb2-d1cf60b12c68 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:49.908931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160728b6-e825-43c4-a58a-51be0135a2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:49.909926 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160728b6-e825-43c4-a58a-51be0135a2c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.910213 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160728b6-e825-43c4-a58a-51be0135a2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.910492 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160728b6-e825-43c4-a58a-51be0135a2c6 admin admin] PATH_INFO: `/v3/projects/7807468a5b804233b963138d3b5d88c0/users/caed2d3f405d4f84a9a778b51364da66/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.926250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160728b6-e825-43c4-a58a-51be0135a2c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7807468a5b804233b963138d3b5d88c0, user_id=caed2d3f405d4f84a9a778b51364da66, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:49.929722 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160728b6-e825-43c4-a58a-51be0135a2c6 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:49.946289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1183/2348] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:49 2022] PUT /identity/v3/projects/7807468a5b804233b963138d3b5d88c0/users/caed2d3f405d4f84a9a778b51364da66/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:49.957758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498707f-6ba5-46b8-84b9-9600d9ec9721 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:49.958056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498707f-6ba5-46b8-84b9-9600d9ec9721 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:49.958338 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b498707f-6ba5-46b8-84b9-9600d9ec9721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:49.987816 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b498707f-6ba5-46b8-84b9-9600d9ec9721 None None] MFA Rules not processed for user `caed2d3f405d4f84a9a778b51364da66`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:49.989067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b498707f-6ba5-46b8-84b9-9600d9ec9721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:50.067910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1166/2349] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:50.080680 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160728b6-e825-43c4-a58a-51be0135a2c6 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.082001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-160728b6-e825-43c4-a58a-51be0135a2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:50.149423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160728b6-e825-43c4-a58a-51be0135a2c6 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.153051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0116b0b-a5b4-4b9c-bab7-80834cc4844c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ff10db6937b4f2b889c30dd5fad8111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.153742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0116b0b-a5b4-4b9c-bab7-80834cc4844c service glance] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.154030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0116b0b-a5b4-4b9c-bab7-80834cc4844c service glance] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.154309 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0116b0b-a5b4-4b9c-bab7-80834cc4844c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.163039 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0116b0b-a5b4-4b9c-bab7-80834cc4844c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:50.164418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0116b0b-a5b4-4b9c-bab7-80834cc4844c service glance] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:50.173809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1184/2350] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:50 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:50.763129 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498707f-6ba5-46b8-84b9-9600d9ec9721 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:50.764546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b498707f-6ba5-46b8-84b9-9600d9ec9721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:50.812736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:50.836178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b498707f-6ba5-46b8-84b9-9600d9ec9721 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:50.840661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:50.841631 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.841922 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.842204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] PATH_INFO: `/v3/users/68c2fdb3a02c42029fd40686ffb8a24a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.843402 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] RBAC: Authorizing `identity:delete_user(user_id=68c2fdb3a02c42029fd40686ffb8a24a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:50.845340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:50.901075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68c2fdb3a02c42029fd40686ffb8a24a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:50.916067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68c2fdb3a02c42029fd40686ffb8a24a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:50.916829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68c2fdb3a02c42029fd40686ffb8a24a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:50.931845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68c2fdb3a02c42029fd40686ffb8a24a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:50.942800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68c2fdb3a02c42029fd40686ffb8a24a'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:50.951526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1167/2351] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:29:50 2022] DELETE /identity/v3/users/68c2fdb3a02c42029fd40686ffb8a24a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:29:50.957194 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09042ee4-5402-4511-96fa-36c5757bef3d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:50.957731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09042ee4-5402-4511-96fa-36c5757bef3d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:50.958342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09042ee4-5402-4511-96fa-36c5757bef3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:50.984445 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-09042ee4-5402-4511-96fa-36c5757bef3d None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:50.986511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-09042ee4-5402-4511-96fa-36c5757bef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:51.032390 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:51.058416 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1185/2352] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:29:51.068174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.069677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:51.133006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6424c2-6489-465e-a587-94416f7d287d admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.136774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b68a78-961e-42c1-903b-d10d60486863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.137472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b68a78-961e-42c1-903b-d10d60486863 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.137761 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b68a78-961e-42c1-903b-d10d60486863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.138038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b68a78-961e-42c1-903b-d10d60486863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.147764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b68a78-961e-42c1-903b-d10d60486863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.149526 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b68a78-961e-42c1-903b-d10d60486863 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.159839 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1168/2353] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:29:51.470427 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09042ee4-5402-4511-96fa-36c5757bef3d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.471794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-09042ee4-5402-4511-96fa-36c5757bef3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:51.484546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09042ee4-5402-4511-96fa-36c5757bef3d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.500455 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.501454 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.502186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.502186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] PATH_INFO: `/v3/projects/a5aab80e3a134ec588ecf32eb1f0a398` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.503353 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5aab80e3a134ec588ecf32eb1f0a398)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.505292 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.560796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5aab80e3a134ec588ecf32eb1f0a398', 'request_id': 'req-3b437607-df4c-4334-8818-94ea2c8f44a1'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:51.568669 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5aab80e3a134ec588ecf32eb1f0a398', 'request_id': 'req-3b437607-df4c-4334-8818-94ea2c8f44a1'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:51.578516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5aab80e3a134ec588ecf32eb1f0a398', 'request_id': 'req-3b437607-df4c-4334-8818-94ea2c8f44a1'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:51.590434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] The token cache is being invalidate because project a5aab80e3a134ec588ecf32eb1f0a398 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:29:51.590731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:51.591366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3b437607-df4c-4334-8818-94ea2c8f44a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:29:51.592928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1186/2354] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:29:51 2022] DELETE /identity/v3/projects/a5aab80e3a134ec588ecf32eb1f0a398 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:29:51.597897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403cc81-5b47-44b0-a1db-2073077d0cd3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.598190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403cc81-5b47-44b0-a1db-2073077d0cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.598499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b403cc81-5b47-44b0-a1db-2073077d0cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.617919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b403cc81-5b47-44b0-a1db-2073077d0cd3 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:51.619263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b403cc81-5b47-44b0-a1db-2073077d0cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:51.692692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1169/2355] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:51.700513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f850-81b5-4f80-8e52-cfe30507597b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.700800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f850-81b5-4f80-8e52-cfe30507597b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.701076 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8053f850-81b5-4f80-8e52-cfe30507597b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.719719 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-8053f850-81b5-4f80-8e52-cfe30507597b None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:51.720935 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8053f850-81b5-4f80-8e52-cfe30507597b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:51.732604 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1187/2356] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:51.736857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403cc81-5b47-44b0-a1db-2073077d0cd3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.743708 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:29:51.748184 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b403cc81-5b47-44b0-a1db-2073077d0cd3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.754304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff83dc9-4ed2-4082-9f81-60bcbb3d6faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.755135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff83dc9-4ed2-4082-9f81-60bcbb3d6faa admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.755483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff83dc9-4ed2-4082-9f81-60bcbb3d6faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.755938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff83dc9-4ed2-4082-9f81-60bcbb3d6faa admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.757525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff83dc9-4ed2-4082-9f81-60bcbb3d6faa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:29:51.757847 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff83dc9-4ed2-4082-9f81-60bcbb3d6faa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.759482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff83dc9-4ed2-4082-9f81-60bcbb3d6faa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.771419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1170/2357] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:29:51.776569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053f850-81b5-4f80-8e52-cfe30507597b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.786048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8053f850-81b5-4f80-8e52-cfe30507597b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.791560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d841426-271b-4dbc-bd48-6ab4dd05c77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.792492 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d841426-271b-4dbc-bd48-6ab4dd05c77d admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.792924 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d841426-271b-4dbc-bd48-6ab4dd05c77d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.793077 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d841426-271b-4dbc-bd48-6ab4dd05c77d admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.793842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d841426-271b-4dbc-bd48-6ab4dd05c77d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.795828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d841426-271b-4dbc-bd48-6ab4dd05c77d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.822360 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1188/2358] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:29:51 2022] POST /identity/v3/projects => generated 365 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:51.826572 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff83dc9-4ed2-4082-9f81-60bcbb3d6faa admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.837805 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff83dc9-4ed2-4082-9f81-60bcbb3d6faa admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.900829 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66064c11-8dd0-41e0-9313-1777f792b1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.901595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66064c11-8dd0-41e0-9313-1777f792b1a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.901830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66064c11-8dd0-41e0-9313-1777f792b1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.902111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66064c11-8dd0-41e0-9313-1777f792b1a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.902814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66064c11-8dd0-41e0-9313-1777f792b1a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.904743 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66064c11-8dd0-41e0-9313-1777f792b1a2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.925946 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1171/2359] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:29:51 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:29:51.930384 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d841426-271b-4dbc-bd48-6ab4dd05c77d admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.940918 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d841426-271b-4dbc-bd48-6ab4dd05c77d admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.945915 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8cca6-9d28-4bdc-9817-7799ccc7028b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.946576 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8cca6-9d28-4bdc-9817-7799ccc7028b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.946861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8cca6-9d28-4bdc-9817-7799ccc7028b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.947138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8cca6-9d28-4bdc-9817-7799ccc7028b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.947867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8cca6-9d28-4bdc-9817-7799ccc7028b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.949634 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8cca6-9d28-4bdc-9817-7799ccc7028b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:51.960563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1189/2360] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:51.964636 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66064c11-8dd0-41e0-9313-1777f792b1a2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:51.974675 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66064c11-8dd0-41e0-9313-1777f792b1a2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:51.980546 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646d5b1-3b4d-4c0d-9709-7f7a13c7709a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:51.981298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d5b1-3b4d-4c0d-9709-7f7a13c7709a admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:51.981701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d5b1-3b4d-4c0d-9709-7f7a13c7709a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:51.981990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646d5b1-3b4d-4c0d-9709-7f7a13c7709a admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:51.982970 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646d5b1-3b4d-4c0d-9709-7f7a13c7709a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:51.984713 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646d5b1-3b4d-4c0d-9709-7f7a13c7709a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.001364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1172/2361] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:29:52.002557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8cca6-9d28-4bdc-9817-7799ccc7028b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.013695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8cca6-9d28-4bdc-9817-7799ccc7028b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.018490 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f163d3-88e2-4827-8c4e-513293bf1a38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.019383 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f163d3-88e2-4827-8c4e-513293bf1a38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.019696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f163d3-88e2-4827-8c4e-513293bf1a38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.019979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f163d3-88e2-4827-8c4e-513293bf1a38 admin admin] PATH_INFO: `/v3/projects/4a041ebd48564b2a8c1ab52cdbac6e34/users/0f4856b9537049d19e9ea95bb1ff1be0/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.036800 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f163d3-88e2-4827-8c4e-513293bf1a38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a041ebd48564b2a8c1ab52cdbac6e34, user_id=0f4856b9537049d19e9ea95bb1ff1be0, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:52.039854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f163d3-88e2-4827-8c4e-513293bf1a38 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.060999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1190/2362] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:29:52 2022] PUT /identity/v3/projects/4a041ebd48564b2a8c1ab52cdbac6e34/users/0f4856b9537049d19e9ea95bb1ff1be0/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:29:52.072250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09753ec9-32c7-4286-8402-ff3260604352 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.072544 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09753ec9-32c7-4286-8402-ff3260604352 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.072848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09753ec9-32c7-4286-8402-ff3260604352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.101655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-09753ec9-32c7-4286-8402-ff3260604352 None None] MFA Rules not processed for user `0f4856b9537049d19e9ea95bb1ff1be0`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:29:52.102897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-09753ec9-32c7-4286-8402-ff3260604352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:52.178706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1173/2363] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:29:52 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:29:52.190226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f163d3-88e2-4827-8c4e-513293bf1a38 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:29:52.192016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e9f163d3-88e2-4827-8c4e-513293bf1a38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:29:52.261203 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f163d3-88e2-4827-8c4e-513293bf1a38 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:29:52.264835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef92b5-2313-45f1-b4e5-8f7e2cfeb183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:29:52.265517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef92b5-2313-45f1-b4e5-8f7e2cfeb183 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:29:52.265808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef92b5-2313-45f1-b4e5-8f7e2cfeb183 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:29:52.266120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef92b5-2313-45f1-b4e5-8f7e2cfeb183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:29:52.275823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef92b5-2313-45f1-b4e5-8f7e2cfeb183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:29:52.277605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef92b5-2313-45f1-b4e5-8f7e2cfeb183 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:29:52.287502 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1191/2364] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:29:52 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:30:02.174842 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:02.283052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:30:14.512527 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854818a3-4e73-4182-a32d-c258b8f6f638 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:30:14.513415 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854818a3-4e73-4182-a32d-c258b8f6f638 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:30:14.513701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854818a3-4e73-4182-a32d-c258b8f6f638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:30:14.533696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-854818a3-4e73-4182-a32d-c258b8f6f638 None None] MFA Rules not processed for user `8f448776d1074e22bfe6cfbe05e01db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:30:14.534972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-854818a3-4e73-4182-a32d-c258b8f6f638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:30:14.599404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1174/2365] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:30:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:30:24.592094 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=25 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:03.326245 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef92b5-2313-45f1-b4e5-8f7e2cfeb183 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:03.328324 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-36ef92b5-2313-45f1-b4e5-8f7e2cfeb183 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:03.394585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef92b5-2313-45f1-b4e5-8f7e2cfeb183 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:03.399139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:03.400026 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:03.400314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:03.400594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] PATH_INFO: `/v3/users/caed2d3f405d4f84a9a778b51364da66` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:03.403001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=caed2d3f405d4f84a9a778b51364da66)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:03.404963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:03.453946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'caed2d3f405d4f84a9a778b51364da66'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:03.461066 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'caed2d3f405d4f84a9a778b51364da66'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:03.461777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'caed2d3f405d4f84a9a778b51364da66'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:03.471985 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'caed2d3f405d4f84a9a778b51364da66'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:03.492399 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'caed2d3f405d4f84a9a778b51364da66'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:03.499620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1192/2366] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:31:03 2022] DELETE /identity/v3/users/caed2d3f405d4f84a9a778b51364da66 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:31:03.504185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c987-7b54-4a46-833a-88af5a5812bc None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:03.504665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c987-7b54-4a46-833a-88af5a5812bc None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:03.504992 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d0c987-7b54-4a46-833a-88af5a5812bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:03.523341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b7d0c987-7b54-4a46-833a-88af5a5812bc None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:03.524559 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b7d0c987-7b54-4a46-833a-88af5a5812bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:03.596605 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1175/2367] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:03.608271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:03.610256 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:03.675967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e023955a-7fda-432f-9c9c-721e658508b7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:03.679605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7a5de-7a17-4216-b38c-24a43b418ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:03.680260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7a5de-7a17-4216-b38c-24a43b418ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:03.680546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7a5de-7a17-4216-b38c-24a43b418ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:03.680825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7a5de-7a17-4216-b38c-24a43b418ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:03.690272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7a5de-7a17-4216-b38c-24a43b418ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:03.691709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7a5de-7a17-4216-b38c-24a43b418ed1 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:03.704343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1193/2368] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:31:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:31:04.003717 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c987-7b54-4a46-833a-88af5a5812bc None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:04.007210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b7d0c987-7b54-4a46-833a-88af5a5812bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:04.020835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d0c987-7b54-4a46-833a-88af5a5812bc None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:04.025684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:04.027245 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.027727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.028174 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] PATH_INFO: `/v3/projects/7807468a5b804233b963138d3b5d88c0` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.029551 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7807468a5b804233b963138d3b5d88c0)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:04.032609 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:04.090840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7807468a5b804233b963138d3b5d88c0', 'request_id': 'req-7b064b01-373a-490d-ba85-b4fde914b1dd'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:04.096875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7807468a5b804233b963138d3b5d88c0', 'request_id': 'req-7b064b01-373a-490d-ba85-b4fde914b1dd'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:04.106747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7807468a5b804233b963138d3b5d88c0', 'request_id': 'req-7b064b01-373a-490d-ba85-b4fde914b1dd'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:04.119958 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] The token cache is being invalidate because project 7807468a5b804233b963138d3b5d88c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:31:04.120903 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:04.121984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7b064b01-373a-490d-ba85-b4fde914b1dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:04.123654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1176/2369] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:31:04 2022] DELETE /identity/v3/projects/7807468a5b804233b963138d3b5d88c0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:31:04.128582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30924418-ff1f-41a4-9abc-dc9b0bed5906 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.128881 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30924418-ff1f-41a4-9abc-dc9b0bed5906 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.129162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30924418-ff1f-41a4-9abc-dc9b0bed5906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.149343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-30924418-ff1f-41a4-9abc-dc9b0bed5906 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:04.150608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-30924418-ff1f-41a4-9abc-dc9b0bed5906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:04.169769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:04.223789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1194/2370] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:31:04.232451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c010e3-5648-4164-8481-6078654d5ff2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.232775 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c010e3-5648-4164-8481-6078654d5ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.233081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c010e3-5648-4164-8481-6078654d5ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.252075 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c0c010e3-5648-4164-8481-6078654d5ff2 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:04.253314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c0c010e3-5648-4164-8481-6078654d5ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:04.261365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1177/2371] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:04.265835 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30924418-ff1f-41a4-9abc-dc9b0bed5906 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:04.275519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30924418-ff1f-41a4-9abc-dc9b0bed5906 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:04.280826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388a74f-7652-4a6e-9da4-bfda910d93ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:04.281515 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388a74f-7652-4a6e-9da4-bfda910d93ad admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.281802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388a74f-7652-4a6e-9da4-bfda910d93ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.282077 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f388a74f-7652-4a6e-9da4-bfda910d93ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.282811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388a74f-7652-4a6e-9da4-bfda910d93ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:31:04.283115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388a74f-7652-4a6e-9da4-bfda910d93ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:04.284713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f388a74f-7652-4a6e-9da4-bfda910d93ad admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:04.305263 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1195/2372] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:31:04.307039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c010e3-5648-4164-8481-6078654d5ff2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:04.317026 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c010e3-5648-4164-8481-6078654d5ff2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:04.321749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e993bcd-434a-49fe-87d7-9f8019ad6ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:04.322404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e993bcd-434a-49fe-87d7-9f8019ad6ea6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.322695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e993bcd-434a-49fe-87d7-9f8019ad6ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.322979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e993bcd-434a-49fe-87d7-9f8019ad6ea6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.323705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e993bcd-434a-49fe-87d7-9f8019ad6ea6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:04.325763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e993bcd-434a-49fe-87d7-9f8019ad6ea6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:04.353512 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1178/2373] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:31:04 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:31:04.358370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388a74f-7652-4a6e-9da4-bfda910d93ad admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:04.368760 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f388a74f-7652-4a6e-9da4-bfda910d93ad admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:04.427733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61d0d5-a917-4638-8598-21fc50a4873e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:04.428428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61d0d5-a917-4638-8598-21fc50a4873e admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.428711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61d0d5-a917-4638-8598-21fc50a4873e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.428997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc61d0d5-a917-4638-8598-21fc50a4873e admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.429736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61d0d5-a917-4638-8598-21fc50a4873e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:04.431681 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc61d0d5-a917-4638-8598-21fc50a4873e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:04.454239 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1196/2374] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:31:04 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:31:04.458522 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e993bcd-434a-49fe-87d7-9f8019ad6ea6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:04.469555 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e993bcd-434a-49fe-87d7-9f8019ad6ea6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:04.474403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040db4e1-e219-4ca2-ae33-f67ad3976b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:04.475319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040db4e1-e219-4ca2-ae33-f67ad3976b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.475794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040db4e1-e219-4ca2-ae33-f67ad3976b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.476265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040db4e1-e219-4ca2-ae33-f67ad3976b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.477081 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040db4e1-e219-4ca2-ae33-f67ad3976b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:04.478858 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040db4e1-e219-4ca2-ae33-f67ad3976b14 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:04.491109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1179/2375] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:31:04.495108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61d0d5-a917-4638-8598-21fc50a4873e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:04.505401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc61d0d5-a917-4638-8598-21fc50a4873e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:04.510067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d766d0e3-8199-4e6d-95f7-4d8e0371b9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:04.510951 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d0e3-8199-4e6d-95f7-4d8e0371b9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.511470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d0e3-8199-4e6d-95f7-4d8e0371b9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.511769 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766d0e3-8199-4e6d-95f7-4d8e0371b9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.512483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d0e3-8199-4e6d-95f7-4d8e0371b9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:04.514106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d766d0e3-8199-4e6d-95f7-4d8e0371b9a7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:04.524337 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1197/2376] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:31:04.528989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040db4e1-e219-4ca2-ae33-f67ad3976b14 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:04.539290 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040db4e1-e219-4ca2-ae33-f67ad3976b14 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:04.545153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99994aa0-2f13-4633-8d68-acd91e869afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:04.546082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99994aa0-2f13-4633-8d68-acd91e869afe admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.546361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99994aa0-2f13-4633-8d68-acd91e869afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.546639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99994aa0-2f13-4633-8d68-acd91e869afe admin admin] PATH_INFO: `/v3/projects/8e6e26c4c8f940e883e9e5548a0318ca/users/51d0218137d049d199521460a0211504/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.561520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99994aa0-2f13-4633-8d68-acd91e869afe admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e6e26c4c8f940e883e9e5548a0318ca, user_id=51d0218137d049d199521460a0211504, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:04.564463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99994aa0-2f13-4633-8d68-acd91e869afe admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:04.578746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1180/2377] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:31:04 2022] PUT /identity/v3/projects/8e6e26c4c8f940e883e9e5548a0318ca/users/51d0218137d049d199521460a0211504/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:31:04.590300 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2f0f7-d415-4f99-a2b8-2ebba4ac1b9d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.590612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2f0f7-d415-4f99-a2b8-2ebba4ac1b9d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.590892 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2f0f7-d415-4f99-a2b8-2ebba4ac1b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.619186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7fc2f0f7-d415-4f99-a2b8-2ebba4ac1b9d None None] MFA Rules not processed for user `51d0218137d049d199521460a0211504`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:04.621056 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7fc2f0f7-d415-4f99-a2b8-2ebba4ac1b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:04.697921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1198/2378] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:04.710387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99994aa0-2f13-4633-8d68-acd91e869afe admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:04.715520 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-99994aa0-2f13-4633-8d68-acd91e869afe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:04.736530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:04.779040 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99994aa0-2f13-4633-8d68-acd91e869afe admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:04.782846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3210b9-0c1d-4393-96f0-a883807046c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:04.783500 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3210b9-0c1d-4393-96f0-a883807046c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:04.783786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3210b9-0c1d-4393-96f0-a883807046c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:04.784109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3210b9-0c1d-4393-96f0-a883807046c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:04.793313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3210b9-0c1d-4393-96f0-a883807046c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:04.794763 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3210b9-0c1d-4393-96f0-a883807046c1 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:04.803519 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1181/2379] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:31:04 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:31:05.791789 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2f0f7-d415-4f99-a2b8-2ebba4ac1b9d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:05.795551 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7fc2f0f7-d415-4f99-a2b8-2ebba4ac1b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:05.819795 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:05.872049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2f0f7-d415-4f99-a2b8-2ebba4ac1b9d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:05.876589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:05.877503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:05.877794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:05.878072 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] PATH_INFO: `/v3/users/0f4856b9537049d19e9ea95bb1ff1be0` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:05.879618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f4856b9537049d19e9ea95bb1ff1be0)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:05.881617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:05.935361 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f4856b9537049d19e9ea95bb1ff1be0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:05.942638 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f4856b9537049d19e9ea95bb1ff1be0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:05.943298 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f4856b9537049d19e9ea95bb1ff1be0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:05.953857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f4856b9537049d19e9ea95bb1ff1be0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:05.966364 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f4856b9537049d19e9ea95bb1ff1be0'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:05.972262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1199/2380] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:31:05 2022] DELETE /identity/v3/users/0f4856b9537049d19e9ea95bb1ff1be0 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:31:05.976908 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf6f8-c3c3-4227-8556-71c40529dc10 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:05.977598 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf6f8-c3c3-4227-8556-71c40529dc10 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:05.977598 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf6f8-c3c3-4227-8556-71c40529dc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:05.996302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-c65bf6f8-c3c3-4227-8556-71c40529dc10 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:05.997574 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c65bf6f8-c3c3-4227-8556-71c40529dc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:06.058759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1182/2381] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:06.069502 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:06.070954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:06.131366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6088ac1-dfba-4c2b-9a16-3f24003b2a64 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:06.135175 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacdac2-dcb6-4a2a-bc46-56745568038d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:06.135923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacdac2-dcb6-4a2a-bc46-56745568038d service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:06.136216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacdac2-dcb6-4a2a-bc46-56745568038d service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:06.136590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bacdac2-dcb6-4a2a-bc46-56745568038d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:06.146310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacdac2-dcb6-4a2a-bc46-56745568038d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:06.147836 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bacdac2-dcb6-4a2a-bc46-56745568038d service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:06.156027 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bafaab-9321-48cb-882d-58ea340bedd0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:06.156321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bafaab-9321-48cb-882d-58ea340bedd0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:06.156603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bafaab-9321-48cb-882d-58ea340bedd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:06.158100 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1200/2382] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:31:06.175024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-04bafaab-9321-48cb-882d-58ea340bedd0 None None] MFA Rules not processed for user `51d0218137d049d199521460a0211504`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:06.176265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-04bafaab-9321-48cb-882d-58ea340bedd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:06.248404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1183/2383] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:31:06.473702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacdac2-dcb6-4a2a-bc46-56745568038d service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:06.476346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2bacdac2-dcb6-4a2a-bc46-56745568038d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:06.501992 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bacdac2-dcb6-4a2a-bc46-56745568038d service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:06.510452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:06.520642 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:06.521264 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:06.521871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] PATH_INFO: `/v3/projects/4a041ebd48564b2a8c1ab52cdbac6e34` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:06.524073 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a041ebd48564b2a8c1ab52cdbac6e34)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:06.528269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:06.585897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:06.606437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a041ebd48564b2a8c1ab52cdbac6e34', 'request_id': 'req-e2816699-9573-46c1-adea-5378d0afed54'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:06.616620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a041ebd48564b2a8c1ab52cdbac6e34', 'request_id': 'req-e2816699-9573-46c1-adea-5378d0afed54'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:06.626926 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a041ebd48564b2a8c1ab52cdbac6e34', 'request_id': 'req-e2816699-9573-46c1-adea-5378d0afed54'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:06.639579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] The token cache is being invalidate because project 4a041ebd48564b2a8c1ab52cdbac6e34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:31:06.639983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:06.640702 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-e2816699-9573-46c1-adea-5378d0afed54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:06.642404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1201/2384] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:31:06 2022] DELETE /identity/v3/projects/4a041ebd48564b2a8c1ab52cdbac6e34 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:31:07.326906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bafaab-9321-48cb-882d-58ea340bedd0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:07.328588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-04bafaab-9321-48cb-882d-58ea340bedd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:07.406797 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bafaab-9321-48cb-882d-58ea340bedd0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:07.415491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:07.476869 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a2dd3f-c930-42d5-8d8a-49385fe7909e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e97119806774c2482a9886b11ab6f70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:07.477593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2dd3f-c930-42d5-8d8a-49385fe7909e service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:07.477853 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2dd3f-c930-42d5-8d8a-49385fe7909e service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:07.478131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a2dd3f-c930-42d5-8d8a-49385fe7909e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:07.480045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-15a2dd3f-c930-42d5-8d8a-49385fe7909e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:07.492020 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2dd3f-c930-42d5-8d8a-49385fe7909e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:07.493826 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a2dd3f-c930-42d5-8d8a-49385fe7909e service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:07.505592 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1184/2385] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:31:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:31:16.635034 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:17.501529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=27 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:24.111367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab82edc-bd90-467f-bca0-2fe6196c1d63 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:24.111876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab82edc-bd90-467f-bca0-2fe6196c1d63 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:24.112003 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab82edc-bd90-467f-bca0-2fe6196c1d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:24.130735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-7ab82edc-bd90-467f-bca0-2fe6196c1d63 None None] MFA Rules not processed for user `d582e9b7a49c4ea6b9a1d6986d6f4163`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:24.131982 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7ab82edc-bd90-467f-bca0-2fe6196c1d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:24.194854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1202/2386] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:33.652381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf480fe-8fb6-44bf-8f63-a5123ee72efb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:33.652894 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf480fe-8fb6-44bf-8f63-a5123ee72efb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:33.652982 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf480fe-8fb6-44bf-8f63-a5123ee72efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:33.676247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7cf480fe-8fb6-44bf-8f63-a5123ee72efb None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:33.677522 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7cf480fe-8fb6-44bf-8f63-a5123ee72efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:33.741796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1185/2387] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:31:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:33.784418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab82edc-bd90-467f-bca0-2fe6196c1d63 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:33.786043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7ab82edc-bd90-467f-bca0-2fe6196c1d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:33.858709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab82edc-bd90-467f-bca0-2fe6196c1d63 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:33.863676 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78af5fb-4d65-422e-8ad2-bee1216f2dc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:33.864323 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78af5fb-4d65-422e-8ad2-bee1216f2dc4 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:33.864608 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78af5fb-4d65-422e-8ad2-bee1216f2dc4 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:33.864884 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78af5fb-4d65-422e-8ad2-bee1216f2dc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:33.874071 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78af5fb-4d65-422e-8ad2-bee1216f2dc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:33.875730 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78af5fb-4d65-422e-8ad2-bee1216f2dc4 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:33.884428 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1203/2388] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:31:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:31:34.446803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840650dc-83aa-4b32-ab05-85abf677808d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:34.447071 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840650dc-83aa-4b32-ab05-85abf677808d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:34.447281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840650dc-83aa-4b32-ab05-85abf677808d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:34.466236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-840650dc-83aa-4b32-ab05-85abf677808d None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:34.467460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-840650dc-83aa-4b32-ab05-85abf677808d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:34.474718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1186/2389] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 12 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:31:34.488135 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78af5fb-4d65-422e-8ad2-bee1216f2dc4 service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:34.499434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78af5fb-4d65-422e-8ad2-bee1216f2dc4 service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:34.504717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f9e4a-4f0b-4c50-a303-6ea00bfe8b27 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:34.505422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f9e4a-4f0b-4c50-a303-6ea00bfe8b27 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:34.505713 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f9e4a-4f0b-4c50-a303-6ea00bfe8b27 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:34.505989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f9e4a-4f0b-4c50-a303-6ea00bfe8b27 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:34.518184 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f9e4a-4f0b-4c50-a303-6ea00bfe8b27 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:34.520226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f9e4a-4f0b-4c50-a303-6ea00bfe8b27 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:34.531270 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1204/2390] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:31:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:31:34.793308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0863ce-b655-4ce3-ab0a-d52aab1903f8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:34.793871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0863ce-b655-4ce3-ab0a-d52aab1903f8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:34.794180 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0863ce-b655-4ce3-ab0a-d52aab1903f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:34.813450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1c0863ce-b655-4ce3-ab0a-d52aab1903f8 None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:34.815082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1c0863ce-b655-4ce3-ab0a-d52aab1903f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:34.823241 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1187/2391] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:34.836557 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f9e4a-4f0b-4c50-a303-6ea00bfe8b27 service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:34.848115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f9e4a-4f0b-4c50-a303-6ea00bfe8b27 service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:34.853474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a7a1c-f388-4adc-8e6b-e24b672431f6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:34.854700 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a7a1c-f388-4adc-8e6b-e24b672431f6 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:34.854986 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a7a1c-f388-4adc-8e6b-e24b672431f6 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:34.855265 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a7a1c-f388-4adc-8e6b-e24b672431f6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:34.864336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a7a1c-f388-4adc-8e6b-e24b672431f6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:34.866002 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342a7a1c-f388-4adc-8e6b-e24b672431f6 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:34.875932 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1205/2392] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:31:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:31:35.258807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b7d79-c783-417b-98e6-caf23106350d None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:35.259101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b7d79-c783-417b-98e6-caf23106350d None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:35.259384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14b7d79-c783-417b-98e6-caf23106350d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:35.280801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b14b7d79-c783-417b-98e6-caf23106350d None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:35.282093 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b14b7d79-c783-417b-98e6-caf23106350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:35.289585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1188/2393] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 12 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:35.302422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a7a1c-f388-4adc-8e6b-e24b672431f6 service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:35.313841 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a7a1c-f388-4adc-8e6b-e24b672431f6 service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:35.319595 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22302501-180f-4611-9401-766c18f129ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:35.320271 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22302501-180f-4611-9401-766c18f129ac service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:35.320560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22302501-180f-4611-9401-766c18f129ac service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:35.320842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22302501-180f-4611-9401-766c18f129ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:35.331639 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22302501-180f-4611-9401-766c18f129ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:35.333530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22302501-180f-4611-9401-766c18f129ac service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:35.353233 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1206/2394] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:31:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:31:37.533046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b7d79-c783-417b-98e6-caf23106350d None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:37.535276 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b14b7d79-c783-417b-98e6-caf23106350d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:37.631046 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14b7d79-c783-417b-98e6-caf23106350d None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:37.636111 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:37.636776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:37.637057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:37.637361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] PATH_INFO: `/v3/users/51d0218137d049d199521460a0211504` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:37.638638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51d0218137d049d199521460a0211504)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:37.640586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:37.699086 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51d0218137d049d199521460a0211504'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:37.710868 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51d0218137d049d199521460a0211504'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:37.712034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51d0218137d049d199521460a0211504'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:37.730144 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51d0218137d049d199521460a0211504'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:37.740082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51d0218137d049d199521460a0211504'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:37.752326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1189/2395] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:31:37 2022] DELETE /identity/v3/users/51d0218137d049d199521460a0211504 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:31:37.756761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbe293-0c17-493d-8483-e4a08f90be73 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:37.757257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbe293-0c17-493d-8483-e4a08f90be73 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:37.757366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cbe293-0c17-493d-8483-e4a08f90be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:37.775363 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e8cbe293-0c17-493d-8483-e4a08f90be73 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:37.779535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e8cbe293-0c17-493d-8483-e4a08f90be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:37.835444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=100 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:37.847613 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1207/2396] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:31:37.858057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:37.861665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:37.868808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:37.927355 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90621fb1-b6d3-4501-9a02-5bdbe4bda5f6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:37.931065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96660f61-9945-45b5-bfa8-8c4da5c25b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:37.931723 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96660f61-9945-45b5-bfa8-8c4da5c25b0f service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:37.932348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96660f61-9945-45b5-bfa8-8c4da5c25b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:37.932770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96660f61-9945-45b5-bfa8-8c4da5c25b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:37.942967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96660f61-9945-45b5-bfa8-8c4da5c25b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:37.944423 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96660f61-9945-45b5-bfa8-8c4da5c25b0f service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:37.955773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1190/2397] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:31:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:31:38.174947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbe293-0c17-493d-8483-e4a08f90be73 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:38.177194 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e8cbe293-0c17-493d-8483-e4a08f90be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:38.190806 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cbe293-0c17-493d-8483-e4a08f90be73 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:38.196063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:38.196957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.197241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.197623 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] PATH_INFO: `/v3/projects/8e6e26c4c8f940e883e9e5548a0318ca` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.199691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e6e26c4c8f940e883e9e5548a0318ca)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:38.202243 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:38.255968 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e6e26c4c8f940e883e9e5548a0318ca', 'request_id': 'req-888a0eeb-b66f-4b89-9285-addd0c65c0be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:38.263217 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e6e26c4c8f940e883e9e5548a0318ca', 'request_id': 'req-888a0eeb-b66f-4b89-9285-addd0c65c0be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:38.272139 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e6e26c4c8f940e883e9e5548a0318ca', 'request_id': 'req-888a0eeb-b66f-4b89-9285-addd0c65c0be'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:38.285872 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] The token cache is being invalidate because project 8e6e26c4c8f940e883e9e5548a0318ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:31:38.286161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:38.286834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-888a0eeb-b66f-4b89-9285-addd0c65c0be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:31:38.288371 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1208/2398] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:31:38 2022] DELETE /identity/v3/projects/8e6e26c4c8f940e883e9e5548a0318ca => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:31:38.294014 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0af31-38f9-4959-8186-f12db175d9b8 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.294319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0af31-38f9-4959-8186-f12db175d9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.294599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc0af31-38f9-4959-8186-f12db175d9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.317661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-1cc0af31-38f9-4959-8186-f12db175d9b8 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:38.318923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-1cc0af31-38f9-4959-8186-f12db175d9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:38.384098 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1191/2399] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:38.392891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93b389-077c-495e-bb4c-a0eb0145740a None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.393186 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93b389-077c-495e-bb4c-a0eb0145740a None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.393508 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff93b389-077c-495e-bb4c-a0eb0145740a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.413138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ff93b389-077c-495e-bb4c-a0eb0145740a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:38.414422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ff93b389-077c-495e-bb4c-a0eb0145740a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:38.423920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1209/2400] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:31:38.428324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0af31-38f9-4959-8186-f12db175d9b8 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:38.440392 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc0af31-38f9-4959-8186-f12db175d9b8 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:38.445002 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c667de9-9519-444d-928d-f58f49439455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:38.445701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c667de9-9519-444d-928d-f58f49439455 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.445994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c667de9-9519-444d-928d-f58f49439455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.446269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c667de9-9519-444d-928d-f58f49439455 admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.447102 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c667de9-9519-444d-928d-f58f49439455 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:31:38.447410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c667de9-9519-444d-928d-f58f49439455 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:38.449191 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c667de9-9519-444d-928d-f58f49439455 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:38.461823 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1192/2401] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:31:38.466413 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93b389-077c-495e-bb4c-a0eb0145740a None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:38.476068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff93b389-077c-495e-bb4c-a0eb0145740a None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:38.481953 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183f9de-a350-43ea-bfc4-7ba3e60c84ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:38.482627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183f9de-a350-43ea-bfc4-7ba3e60c84ca admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.482911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183f9de-a350-43ea-bfc4-7ba3e60c84ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.483187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d183f9de-a350-43ea-bfc4-7ba3e60c84ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.483891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183f9de-a350-43ea-bfc4-7ba3e60c84ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:38.485853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d183f9de-a350-43ea-bfc4-7ba3e60c84ca admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:38.513864 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1210/2402] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:31:38 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:31:38.518035 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c667de9-9519-444d-928d-f58f49439455 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:38.528846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c667de9-9519-444d-928d-f58f49439455 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:38.591806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e11c0f-8ff2-4198-af31-f7b005b6b703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:38.592640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e11c0f-8ff2-4198-af31-f7b005b6b703 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.593026 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e11c0f-8ff2-4198-af31-f7b005b6b703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.593426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e11c0f-8ff2-4198-af31-f7b005b6b703 admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.594243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e11c0f-8ff2-4198-af31-f7b005b6b703 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:38.596365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e11c0f-8ff2-4198-af31-f7b005b6b703 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:38.624939 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1193/2403] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:31:38 2022] POST /identity/v3/users => generated 296 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:31:38.629474 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183f9de-a350-43ea-bfc4-7ba3e60c84ca admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:38.641988 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d183f9de-a350-43ea-bfc4-7ba3e60c84ca admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:38.646717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92015305-8fd9-4b01-a8e1-4956015e9ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:38.647425 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92015305-8fd9-4b01-a8e1-4956015e9ff2 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.647714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92015305-8fd9-4b01-a8e1-4956015e9ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.647987 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92015305-8fd9-4b01-a8e1-4956015e9ff2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.648877 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92015305-8fd9-4b01-a8e1-4956015e9ff2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:38.650542 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92015305-8fd9-4b01-a8e1-4956015e9ff2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:38.663591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1211/2404] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:31:38.668361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e11c0f-8ff2-4198-af31-f7b005b6b703 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:38.684751 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e11c0f-8ff2-4198-af31-f7b005b6b703 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:38.690255 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9698c1-2e2c-4221-8e01-9e23a9011789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:38.690952 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9698c1-2e2c-4221-8e01-9e23a9011789 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.691248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9698c1-2e2c-4221-8e01-9e23a9011789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.691536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9698c1-2e2c-4221-8e01-9e23a9011789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.692282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9698c1-2e2c-4221-8e01-9e23a9011789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:38.693938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9698c1-2e2c-4221-8e01-9e23a9011789 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:38.706302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1194/2405] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:31:38.711573 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92015305-8fd9-4b01-a8e1-4956015e9ff2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:38.723393 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92015305-8fd9-4b01-a8e1-4956015e9ff2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:38.728620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67690e-0484-4a38-82ff-8c1d3519157e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:38.730432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67690e-0484-4a38-82ff-8c1d3519157e admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.730730 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67690e-0484-4a38-82ff-8c1d3519157e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.731059 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67690e-0484-4a38-82ff-8c1d3519157e admin admin] PATH_INFO: `/v3/projects/15d8ac81514645debaeacad3820f81cd/users/09b706fa63194739990a88f4d13533d2/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.748395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67690e-0484-4a38-82ff-8c1d3519157e admin admin] RBAC: Authorizing `identity:create_grant(project_id=15d8ac81514645debaeacad3820f81cd, user_id=09b706fa63194739990a88f4d13533d2, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:38.751234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67690e-0484-4a38-82ff-8c1d3519157e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:38.768818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1212/2406] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:31:38 2022] PUT /identity/v3/projects/15d8ac81514645debaeacad3820f81cd/users/09b706fa63194739990a88f4d13533d2/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:31:38.780748 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9802-53b2-4180-85e7-de37b2932cc2 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:38.780961 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9802-53b2-4180-85e7-de37b2932cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:38.781257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9802-53b2-4180-85e7-de37b2932cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:38.816733 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:38.820476 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b0fd9802-53b2-4180-85e7-de37b2932cc2 None None] MFA Rules not processed for user `09b706fa63194739990a88f4d13533d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:31:38.823195 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b0fd9802-53b2-4180-85e7-de37b2932cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:38.901896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1195/2407] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:31:38.913255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67690e-0484-4a38-82ff-8c1d3519157e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:31:38.915295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1a67690e-0484-4a38-82ff-8c1d3519157e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:31:38.996455 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67690e-0484-4a38-82ff-8c1d3519157e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:31:39.000405 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dcd11-c624-473d-8699-733c05ea9978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:31:39.001063 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dcd11-c624-473d-8699-733c05ea9978 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:31:39.001348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dcd11-c624-473d-8699-733c05ea9978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:31:39.001650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931dcd11-c624-473d-8699-733c05ea9978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:31:39.011198 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931dcd11-c624-473d-8699-733c05ea9978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:31:39.012657 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931dcd11-c624-473d-8699-733c05ea9978 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:31:39.022200 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1213/2408] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:31:38 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:31:48.897897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=26 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:31:49.017972 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=82,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:03.001007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da972fb-8313-448a-8c49-8d7e661ff0d5 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:03.001531 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da972fb-8313-448a-8c49-8d7e661ff0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:03.001715 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da972fb-8313-448a-8c49-8d7e661ff0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:03.019712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-4da972fb-8313-448a-8c49-8d7e661ff0d5 None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:03.020955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4da972fb-8313-448a-8c49-8d7e661ff0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:03.083425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1196/2409] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:32:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:32:03.096704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dcd11-c624-473d-8699-733c05ea9978 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:03.098159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-931dcd11-c624-473d-8699-733c05ea9978 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:03.176571 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931dcd11-c624-473d-8699-733c05ea9978 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:03.184133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5b87c-2810-4a50-a965-a027f9baed20 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:03.184792 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5b87c-2810-4a50-a965-a027f9baed20 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:03.185074 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5b87c-2810-4a50-a965-a027f9baed20 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:03.185348 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5b87c-2810-4a50-a965-a027f9baed20 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:03.194720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5b87c-2810-4a50-a965-a027f9baed20 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:03.196257 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5b87c-2810-4a50-a965-a027f9baed20 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:03.205854 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1214/2410] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:32:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:32:06.820083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da972fb-8313-448a-8c49-8d7e661ff0d5 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:06.945491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da972fb-8313-448a-8c49-8d7e661ff0d5 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:06.954053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:06.956264 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:06.956809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:06.957300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] PATH_INFO: `/v3/users/09b706fa63194739990a88f4d13533d2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:06.958882 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b706fa63194739990a88f4d13533d2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:06.960725 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:07.059209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b706fa63194739990a88f4d13533d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:07.074079 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b706fa63194739990a88f4d13533d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:07.077752 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b706fa63194739990a88f4d13533d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:07.103419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b706fa63194739990a88f4d13533d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:07.119571 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b706fa63194739990a88f4d13533d2'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:07.131978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1197/2411] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:32:06 2022] DELETE /identity/v3/users/09b706fa63194739990a88f4d13533d2 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:32:07.134460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8561-a781-486f-802e-fea7b0d5c41f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:07.135418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8561-a781-486f-802e-fea7b0d5c41f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:07.135949 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe8561-a781-486f-802e-fea7b0d5c41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:07.170397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-41fe8561-a781-486f-802e-fea7b0d5c41f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:07.171796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-41fe8561-a781-486f-802e-fea7b0d5c41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:07.291287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1215/2412] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:32:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:32:07.304932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:07.306445 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:07.348431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:07.400595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5bd43-cca9-49b0-9389-383b5ec55883 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:07.405175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b0836-e530-4cdb-af4c-44371bfdd361 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:07.405884 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b0836-e530-4cdb-af4c-44371bfdd361 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:07.406170 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b0836-e530-4cdb-af4c-44371bfdd361 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:07.406450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60b0836-e530-4cdb-af4c-44371bfdd361 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:07.421223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b0836-e530-4cdb-af4c-44371bfdd361 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:07.423950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60b0836-e530-4cdb-af4c-44371bfdd361 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:07.442732 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1198/2413] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:32:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:32:07.980971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8561-a781-486f-802e-fea7b0d5c41f None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:07.982862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-41fe8561-a781-486f-802e-fea7b0d5c41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:08.012848 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe8561-a781-486f-802e-fea7b0d5c41f None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:08.018824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:08.019738 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:08.020025 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:08.020358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] PATH_INFO: `/v3/projects/15d8ac81514645debaeacad3820f81cd` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:08.022130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15d8ac81514645debaeacad3820f81cd)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:08.024840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:08.087934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15d8ac81514645debaeacad3820f81cd', 'request_id': 'req-3c034257-24b2-497b-9ea9-910f81bd9853'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:08.095837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15d8ac81514645debaeacad3820f81cd', 'request_id': 'req-3c034257-24b2-497b-9ea9-910f81bd9853'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:08.107678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15d8ac81514645debaeacad3820f81cd', 'request_id': 'req-3c034257-24b2-497b-9ea9-910f81bd9853'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:08.126326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] The token cache is being invalidate because project 15d8ac81514645debaeacad3820f81cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:32:08.126776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:08.127698 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:32:08.129966 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1216/2414] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:32:07 2022] DELETE /identity/v3/projects/15d8ac81514645debaeacad3820f81cd => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:32:15.159070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b0836-e530-4cdb-af4c-44371bfdd361 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:15.160578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-c60b0836-e530-4cdb-af4c-44371bfdd361 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:15.173141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60b0836-e530-4cdb-af4c-44371bfdd361 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:15.249563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:15.250223 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:15.250510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:15.250792 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:15.252874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:15.327965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:15.329308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:15.341193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1199/2415] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:32:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:32:16.523079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:16.524540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:16.540172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c034257-24b2-497b-9ea9-910f81bd9853 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:16.547711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d675a5d0-28d6-46b5-85af-b987f47afa16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:16.548388 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675a5d0-28d6-46b5-85af-b987f47afa16 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:16.548676 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675a5d0-28d6-46b5-85af-b987f47afa16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:16.548957 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d675a5d0-28d6-46b5-85af-b987f47afa16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:16.550921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d675a5d0-28d6-46b5-85af-b987f47afa16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:16.572912 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675a5d0-28d6-46b5-85af-b987f47afa16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:16.574993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d675a5d0-28d6-46b5-85af-b987f47afa16 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:16.581883 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:16.591247 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1217/2416] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:32:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:32:23.810154 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:23.811583 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:23.825729 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802a32f-be37-4aa3-9742-078ffddcee2e service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:23.831305 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a64cb0-e50b-4f5a-adda-7d6b808c68cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:23.831947 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a64cb0-e50b-4f5a-adda-7d6b808c68cc service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:23.832234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a64cb0-e50b-4f5a-adda-7d6b808c68cc service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:23.832515 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a64cb0-e50b-4f5a-adda-7d6b808c68cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:23.834253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-73a64cb0-e50b-4f5a-adda-7d6b808c68cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:23.896691 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a64cb0-e50b-4f5a-adda-7d6b808c68cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:23.898267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a64cb0-e50b-4f5a-adda-7d6b808c68cc service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:23.908665 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1200/2417] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:32:24.655189 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef453a-97b8-4e2b-9a1a-ac8cf3eb3f49 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.655794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef453a-97b8-4e2b-9a1a-ac8cf3eb3f49 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.655794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ef453a-97b8-4e2b-9a1a-ac8cf3eb3f49 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.657171 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1218/2418] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:32:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:24.663528 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03d5e3-264a-47cc-9e3f-74f7f0ff529e None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.663925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03d5e3-264a-47cc-9e3f-74f7f0ff529e None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.664304 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03d5e3-264a-47cc-9e3f-74f7f0ff529e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.678122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:24.685611 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-ea03d5e3-264a-47cc-9e3f-74f7f0ff529e None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:24.687383 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ea03d5e3-264a-47cc-9e3f-74f7f0ff529e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:24.697084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1201/2419] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:32:24.709684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef453a-97b8-4e2b-9a1a-ac8cf3eb3f49 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:24.711516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-42ef453a-97b8-4e2b-9a1a-ac8cf3eb3f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:24.790437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ef453a-97b8-4e2b-9a1a-ac8cf3eb3f49 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:24.805527 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d33625-9d67-40da-9368-583eae4cfd0c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ff10db6937b4f2b889c30dd5fad8111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:24.806285 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d33625-9d67-40da-9368-583eae4cfd0c service glance] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.806570 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d33625-9d67-40da-9368-583eae4cfd0c service glance] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.806850 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d33625-9d67-40da-9368-583eae4cfd0c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.818579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d33625-9d67-40da-9368-583eae4cfd0c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:24.820016 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d33625-9d67-40da-9368-583eae4cfd0c service glance] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:24.832747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1219/2420] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:32:24.903023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea06c3-064f-45f0-83bb-9643875770d2 None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.903512 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea06c3-064f-45f0-83bb-9643875770d2 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.903712 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea06c3-064f-45f0-83bb-9643875770d2 None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.905242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1202/2421] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:32:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:24.910491 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f3d90-2c6a-4c1c-b849-dcb20019de97 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.910876 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f3d90-2c6a-4c1c-b849-dcb20019de97 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.911225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f3d90-2c6a-4c1c-b849-dcb20019de97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.929410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1d5f3d90-2c6a-4c1c-b849-dcb20019de97 None None] MFA Rules not processed for user `83837e4f8f9b4e538658876cfb56db16`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:24.930720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1d5f3d90-2c6a-4c1c-b849-dcb20019de97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:24.937530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1220/2422] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:32:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:32:24.949867 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea06c3-064f-45f0-83bb-9643875770d2 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:24.951624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-12ea06c3-064f-45f0-83bb-9643875770d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:24.967227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea06c3-064f-45f0-83bb-9643875770d2 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:24.971594 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14ba433-fa76-4189-abe6-0877eafa8b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:24.972244 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ba433-fa76-4189-abe6-0877eafa8b0f service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:24.972532 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ba433-fa76-4189-abe6-0877eafa8b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:24.972813 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ba433-fa76-4189-abe6-0877eafa8b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:24.981341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ba433-fa76-4189-abe6-0877eafa8b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:24.982815 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14ba433-fa76-4189-abe6-0877eafa8b0f service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:24.991648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1203/2423] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:32:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:32:25.089279 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829da355-ec3e-4ec4-ae86-a1dccad4962b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:25.090546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829da355-ec3e-4ec4-ae86-a1dccad4962b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:25.090871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829da355-ec3e-4ec4-ae86-a1dccad4962b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:25.094874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1221/2424] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:32:25 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:32:25.105172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ecda2-9427-4904-9150-952d9cff3578 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:25.105645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ecda2-9427-4904-9150-952d9cff3578 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:25.106159 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8ecda2-9427-4904-9150-952d9cff3578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:25.136343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5d8ecda2-9427-4904-9150-952d9cff3578 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:25.137722 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5d8ecda2-9427-4904-9150-952d9cff3578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:25.147382 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1204/2425] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:32:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:32:25.162733 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829da355-ec3e-4ec4-ae86-a1dccad4962b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:25.180274 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829da355-ec3e-4ec4-ae86-a1dccad4962b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:25.184615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53786a-c72b-42f5-99f0-321716324dfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:25.185268 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53786a-c72b-42f5-99f0-321716324dfb service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:25.185586 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53786a-c72b-42f5-99f0-321716324dfb service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:25.185861 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b53786a-c72b-42f5-99f0-321716324dfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:25.196919 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53786a-c72b-42f5-99f0-321716324dfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:25.199221 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b53786a-c72b-42f5-99f0-321716324dfb service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:25.216110 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1222/2426] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:32:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:32:29.934239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424402-231c-4a3b-a51e-2508011e2f87 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:29.934740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424402-231c-4a3b-a51e-2508011e2f87 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:29.934834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67424402-231c-4a3b-a51e-2508011e2f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:29.956335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-67424402-231c-4a3b-a51e-2508011e2f87 None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:32:29.957614 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-67424402-231c-4a3b-a51e-2508011e2f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:30.053280 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1205/2427] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:32:30.069091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53786a-c72b-42f5-99f0-321716324dfb service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:32:30.070816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-7b53786a-c72b-42f5-99f0-321716324dfb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:32:30.148660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b53786a-c72b-42f5-99f0-321716324dfb service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:32:30.154294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa0c9f-0487-48fb-8dde-dbd675c3094c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:32:30.154949 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa0c9f-0487-48fb-8dde-dbd675c3094c service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:32:30.155242 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa0c9f-0487-48fb-8dde-dbd675c3094c service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:32:30.155521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa0c9f-0487-48fb-8dde-dbd675c3094c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:32:30.163983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa0c9f-0487-48fb-8dde-dbd675c3094c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:32:30.165633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa0c9f-0487-48fb-8dde-dbd675c3094c service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:32:30.174583 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1223/2428] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:32:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 12 13:32:40.042044 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=34 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:32:40.170846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=67 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:01.376418 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424402-231c-4a3b-a51e-2508011e2f87 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:01.378068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-67424402-231c-4a3b-a51e-2508011e2f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:01.453425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67424402-231c-4a3b-a51e-2508011e2f87 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:01.458491 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:01.459385 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:01.459670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:01.459985 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] PATH_INFO: `/v3/users/d582e9b7a49c4ea6b9a1d6986d6f4163` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:01.461219 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d582e9b7a49c4ea6b9a1d6986d6f4163)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:01.463256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:01.513109 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd582e9b7a49c4ea6b9a1d6986d6f4163'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.520108 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd582e9b7a49c4ea6b9a1d6986d6f4163'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.520793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd582e9b7a49c4ea6b9a1d6986d6f4163'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.532800 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd582e9b7a49c4ea6b9a1d6986d6f4163'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.540570 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-5d6e8e04-4698-4312-a44a-30895eb9e563 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd582e9b7a49c4ea6b9a1d6986d6f4163'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.547794 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1206/2429] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:33:01 2022] DELETE /identity/v3/users/d582e9b7a49c4ea6b9a1d6986d6f4163 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:33:01.731824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa0c9f-0487-48fb-8dde-dbd675c3094c service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:01.744005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa0c9f-0487-48fb-8dde-dbd675c3094c service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:01.809331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:01.810163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:01.810446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:01.810720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] PATH_INFO: `/v3/projects/86acb815cd62409d9856563af3037d38` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:01.812089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86acb815cd62409d9856563af3037d38)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:01.814048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:01.863235 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86acb815cd62409d9856563af3037d38', 'request_id': 'req-9b796853-778a-43ab-8b26-526dee21f582'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.873523 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86acb815cd62409d9856563af3037d38', 'request_id': 'req-9b796853-778a-43ab-8b26-526dee21f582'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.883375 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86acb815cd62409d9856563af3037d38', 'request_id': 'req-9b796853-778a-43ab-8b26-526dee21f582'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.894976 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] The token cache is being invalidate because project 86acb815cd62409d9856563af3037d38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:33:01.895261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.895905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-9b796853-778a-43ab-8b26-526dee21f582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:33:01.897470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1224/2430] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:33:01 2022] DELETE /identity/v3/projects/86acb815cd62409d9856563af3037d38 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:33:01.903117 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274efad-dbd2-47c8-af78-fd1b5ab580cb None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:01.903412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274efad-dbd2-47c8-af78-fd1b5ab580cb None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:01.903692 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3274efad-dbd2-47c8-af78-fd1b5ab580cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:01.921973 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-3274efad-dbd2-47c8-af78-fd1b5ab580cb None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:01.923185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3274efad-dbd2-47c8-af78-fd1b5ab580cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:01.983262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1207/2431] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:01.991230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f0673-7def-45de-8c47-023ebb3a4308 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:01.991519 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f0673-7def-45de-8c47-023ebb3a4308 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:01.991793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f0673-7def-45de-8c47-023ebb3a4308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:02.010986 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c48f0673-7def-45de-8c47-023ebb3a4308 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:02.012272 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c48f0673-7def-45de-8c47-023ebb3a4308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:02.020262 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1225/2432] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:33:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:02.024777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274efad-dbd2-47c8-af78-fd1b5ab580cb None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:02.035019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3274efad-dbd2-47c8-af78-fd1b5ab580cb None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:02.043754 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103b7f4-f16d-4970-bfc6-a9e01cd3fe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:02.044588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103b7f4-f16d-4970-bfc6-a9e01cd3fe4c admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:02.044879 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103b7f4-f16d-4970-bfc6-a9e01cd3fe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:02.045160 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b103b7f4-f16d-4970-bfc6-a9e01cd3fe4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:02.046006 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103b7f4-f16d-4970-bfc6-a9e01cd3fe4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:33:02.046238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103b7f4-f16d-4970-bfc6-a9e01cd3fe4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:02.047835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b103b7f4-f16d-4970-bfc6-a9e01cd3fe4c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:02.060529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1208/2433] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:33:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:33:02.065065 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f0673-7def-45de-8c47-023ebb3a4308 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:02.075703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f0673-7def-45de-8c47-023ebb3a4308 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:02.080395 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb181214-5b1c-44d6-aaad-88e2d15253d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:02.081052 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb181214-5b1c-44d6-aaad-88e2d15253d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:02.081343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb181214-5b1c-44d6-aaad-88e2d15253d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:02.081660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb181214-5b1c-44d6-aaad-88e2d15253d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:02.082366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb181214-5b1c-44d6-aaad-88e2d15253d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:02.084281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb181214-5b1c-44d6-aaad-88e2d15253d8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:02.111524 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1226/2434] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:33:02 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:33:02.115731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103b7f4-f16d-4970-bfc6-a9e01cd3fe4c admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:02.126793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b103b7f4-f16d-4970-bfc6-a9e01cd3fe4c admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:02.130582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:02.185878 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c7de-b109-4b2f-bdb2-0fe8667725fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:02.186602 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c7de-b109-4b2f-bdb2-0fe8667725fc admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:02.186891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c7de-b109-4b2f-bdb2-0fe8667725fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:02.187171 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9c7de-b109-4b2f-bdb2-0fe8667725fc admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:02.187885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9c7de-b109-4b2f-bdb2-0fe8667725fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:02.190179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9c7de-b109-4b2f-bdb2-0fe8667725fc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:02.214345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1209/2435] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:33:02 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:33:02.215754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb181214-5b1c-44d6-aaad-88e2d15253d8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:02.226112 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb181214-5b1c-44d6-aaad-88e2d15253d8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:02.230936 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a6950-4837-4b39-b5ba-0f093ad0f7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:02.231711 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a6950-4837-4b39-b5ba-0f093ad0f7d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:02.231997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a6950-4837-4b39-b5ba-0f093ad0f7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:02.232277 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1a6950-4837-4b39-b5ba-0f093ad0f7d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:02.233013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a6950-4837-4b39-b5ba-0f093ad0f7d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:02.234661 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1a6950-4837-4b39-b5ba-0f093ad0f7d8 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:02.245563 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1227/2436] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:33:02.250620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c7de-b109-4b2f-bdb2-0fe8667725fc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:02.260674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9c7de-b109-4b2f-bdb2-0fe8667725fc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:02.266217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da5d0c8-cebd-419a-9408-a4adf6bd6946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:02.266891 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5d0c8-cebd-419a-9408-a4adf6bd6946 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:02.267176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5d0c8-cebd-419a-9408-a4adf6bd6946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:02.267452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da5d0c8-cebd-419a-9408-a4adf6bd6946 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:02.268184 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5d0c8-cebd-419a-9408-a4adf6bd6946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:02.269812 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da5d0c8-cebd-419a-9408-a4adf6bd6946 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:02.280604 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1210/2437] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:33:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:33:02.284705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a6950-4837-4b39-b5ba-0f093ad0f7d8 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:02.296641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1a6950-4837-4b39-b5ba-0f093ad0f7d8 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:02.301325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0538d4a-4fac-4708-bea4-30d4b1792b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:02.302304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538d4a-4fac-4708-bea4-30d4b1792b41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:02.302590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538d4a-4fac-4708-bea4-30d4b1792b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:02.302867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0538d4a-4fac-4708-bea4-30d4b1792b41 admin admin] PATH_INFO: `/v3/projects/b3897c86b8c847ea9091f3c7ad344fa1/users/8aa0b8a0b7ff4eb0b13fb985dc0d710a/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:02.318794 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0538d4a-4fac-4708-bea4-30d4b1792b41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3897c86b8c847ea9091f3c7ad344fa1, user_id=8aa0b8a0b7ff4eb0b13fb985dc0d710a, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:02.321433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0538d4a-4fac-4708-bea4-30d4b1792b41 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:02.335928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1228/2438] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:33:02 2022] PUT /identity/v3/projects/b3897c86b8c847ea9091f3c7ad344fa1/users/8aa0b8a0b7ff4eb0b13fb985dc0d710a/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:33:02.346577 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851146d-3aff-4887-9f06-b94d87463b1f None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:02.346865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851146d-3aff-4887-9f06-b94d87463b1f None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:02.347146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0851146d-3aff-4887-9f06-b94d87463b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:02.366357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-0851146d-3aff-4887-9f06-b94d87463b1f None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:02.367588 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0851146d-3aff-4887-9f06-b94d87463b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:02.437671 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1211/2439] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:33:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:33:02.447775 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0538d4a-4fac-4708-bea4-30d4b1792b41 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:02.450354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-a0538d4a-4fac-4708-bea4-30d4b1792b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:02.485712 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:02.519192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0538d4a-4fac-4708-bea4-30d4b1792b41 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:02.522893 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75199eb6-2ca2-4596-b064-16ba50b0908a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:02.523675 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75199eb6-2ca2-4596-b064-16ba50b0908a service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:02.523971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75199eb6-2ca2-4596-b064-16ba50b0908a service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:02.524250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75199eb6-2ca2-4596-b064-16ba50b0908a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:02.534326 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75199eb6-2ca2-4596-b064-16ba50b0908a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:02.535781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75199eb6-2ca2-4596-b064-16ba50b0908a service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:02.546244 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1229/2440] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:03.541920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851146d-3aff-4887-9f06-b94d87463b1f None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:03.543606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-0851146d-3aff-4887-9f06-b94d87463b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:03.559281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0851146d-3aff-4887-9f06-b94d87463b1f None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:03.563325 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:03.564135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:03.564522 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:03.565162 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:03.567409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:03.579680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:03.581163 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:03.591399 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1212/2441] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:05.456533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16759fe7-3a2b-4ba0-ab2b-fd4b3f62239f None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:05.457091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16759fe7-3a2b-4ba0-ab2b-fd4b3f62239f None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:05.457091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16759fe7-3a2b-4ba0-ab2b-fd4b3f62239f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:05.477699 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-16759fe7-3a2b-4ba0-ab2b-fd4b3f62239f None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:05.479088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-16759fe7-3a2b-4ba0-ab2b-fd4b3f62239f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:05.554078 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1230/2442] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 12 13:33:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:05.583771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:05.585323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:05.678562 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb5626-31e3-4dce-9d0e-769445442972 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:05.685593 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb17b31-3e45-4d51-94c2-2567f18d0eda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:05.686384 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17b31-3e45-4d51-94c2-2567f18d0eda service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:05.686669 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17b31-3e45-4d51-94c2-2567f18d0eda service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:05.687403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb17b31-3e45-4d51-94c2-2567f18d0eda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:05.700575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb17b31-3e45-4d51-94c2-2567f18d0eda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:05.702312 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb17b31-3e45-4d51-94c2-2567f18d0eda service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:05.722021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1213/2443] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:33:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:33:07.569147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280326d-15ca-49b2-9f19-13fc3b57f020 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:07.569662 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280326d-15ca-49b2-9f19-13fc3b57f020 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:07.569757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280326d-15ca-49b2-9f19-13fc3b57f020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:07.599897 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-1280326d-15ca-49b2-9f19-13fc3b57f020 None None] MFA Rules not processed for user `8aa0b8a0b7ff4eb0b13fb985dc0d710a`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:07.601057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-1280326d-15ca-49b2-9f19-13fc3b57f020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:07.673446 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1231/2444] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:33:07 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:07.677976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb17b31-3e45-4d51-94c2-2567f18d0eda service swift] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:07.691021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:07.694999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb17b31-3e45-4d51-94c2-2567f18d0eda service swift] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:07.705059 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77f2fd-6574-49a4-8d6c-510bd9f0ae4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:07.705861 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77f2fd-6574-49a4-8d6c-510bd9f0ae4a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:07.706352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77f2fd-6574-49a4-8d6c-510bd9f0ae4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:07.706701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77f2fd-6574-49a4-8d6c-510bd9f0ae4a admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:07.707679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77f2fd-6574-49a4-8d6c-510bd9f0ae4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:07.710434 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc77f2fd-6574-49a4-8d6c-510bd9f0ae4a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:07.739918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1214/2445] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:33:07 2022] POST /identity/v3/projects => generated 377 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:33:07.743788 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280326d-15ca-49b2-9f19-13fc3b57f020 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:07.754908 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280326d-15ca-49b2-9f19-13fc3b57f020 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:07.816027 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf52b2b-8ac4-45d1-a357-1e7103e46d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:07.816708 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf52b2b-8ac4-45d1-a357-1e7103e46d8e admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:07.816993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf52b2b-8ac4-45d1-a357-1e7103e46d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:07.817269 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf52b2b-8ac4-45d1-a357-1e7103e46d8e admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:07.818007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf52b2b-8ac4-45d1-a357-1e7103e46d8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:07.819944 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf52b2b-8ac4-45d1-a357-1e7103e46d8e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:07.830187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:07.844707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1232/2446] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:33:07 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:33:07.849670 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77f2fd-6574-49a4-8d6c-510bd9f0ae4a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:07.861906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77f2fd-6574-49a4-8d6c-510bd9f0ae4a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:07.866460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022ea3e-5073-48ae-bffe-3cff27ef5ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:07.867121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022ea3e-5073-48ae-bffe-3cff27ef5ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:07.867407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022ea3e-5073-48ae-bffe-3cff27ef5ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:07.867684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022ea3e-5073-48ae-bffe-3cff27ef5ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:07.868414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022ea3e-5073-48ae-bffe-3cff27ef5ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:07.870070 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022ea3e-5073-48ae-bffe-3cff27ef5ec0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:07.881407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1215/2447] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:33:07.885903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf52b2b-8ac4-45d1-a357-1e7103e46d8e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:07.898286 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf52b2b-8ac4-45d1-a357-1e7103e46d8e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:07.904973 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30480e-e042-413c-b94e-9e699b3365e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:07.905938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30480e-e042-413c-b94e-9e699b3365e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:07.906219 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30480e-e042-413c-b94e-9e699b3365e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:07.906559 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30480e-e042-413c-b94e-9e699b3365e3 admin admin] PATH_INFO: `/v3/projects/a994448bc92a4b1d989752237be923ee/users/20048d10d7f34d43b8865218791c2eda/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:07.924940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30480e-e042-413c-b94e-9e699b3365e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a994448bc92a4b1d989752237be923ee, user_id=20048d10d7f34d43b8865218791c2eda, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:07.927802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30480e-e042-413c-b94e-9e699b3365e3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:07.943991 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1233/2448] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:33:07 2022] PUT /identity/v3/projects/a994448bc92a4b1d989752237be923ee/users/20048d10d7f34d43b8865218791c2eda/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:33:07.947971 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022ea3e-5073-48ae-bffe-3cff27ef5ec0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:07.958959 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022ea3e-5073-48ae-bffe-3cff27ef5ec0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:08.021201 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b5f70-4801-4aa7-a377-1b05194e97fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:08.021917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b5f70-4801-4aa7-a377-1b05194e97fa admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:08.022208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b5f70-4801-4aa7-a377-1b05194e97fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:08.022490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b5f70-4801-4aa7-a377-1b05194e97fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:08.023226 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b5f70-4801-4aa7-a377-1b05194e97fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:08.024833 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b5f70-4801-4aa7-a377-1b05194e97fa admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:08.035753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1216/2449] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:33:07 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:33:08.040169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30480e-e042-413c-b94e-9e699b3365e3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:08.050981 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30480e-e042-413c-b94e-9e699b3365e3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:08.056066 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b66396-ef6b-4489-81b7-93e67bf9eff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:08.056993 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b66396-ef6b-4489-81b7-93e67bf9eff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:08.057280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b66396-ef6b-4489-81b7-93e67bf9eff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:08.057584 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b66396-ef6b-4489-81b7-93e67bf9eff2 admin admin] PATH_INFO: `/v3/projects/a994448bc92a4b1d989752237be923ee/users/20048d10d7f34d43b8865218791c2eda/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:08.060917 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b66396-ef6b-4489-81b7-93e67bf9eff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a994448bc92a4b1d989752237be923ee, user_id=20048d10d7f34d43b8865218791c2eda, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:08.063624 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b66396-ef6b-4489-81b7-93e67bf9eff2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:08.077275 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1234/2450] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:33:08 2022] PUT /identity/v3/projects/a994448bc92a4b1d989752237be923ee/users/20048d10d7f34d43b8865218791c2eda/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:33:12.900469 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2750dc-9f54-4d24-acb4-df97d8aa17a0 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:12.901010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2750dc-9f54-4d24-acb4-df97d8aa17a0 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:12.901010 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2750dc-9f54-4d24-acb4-df97d8aa17a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:12.932016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2e2750dc-9f54-4d24-acb4-df97d8aa17a0 None None] MFA Rules not processed for user `20048d10d7f34d43b8865218791c2eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:12.934024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2e2750dc-9f54-4d24-acb4-df97d8aa17a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:13.035267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1217/2451] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:33:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:13.052926 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b66396-ef6b-4489-81b7-93e67bf9eff2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:13.054948 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b3b66396-ef6b-4489-81b7-93e67bf9eff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:13.159301 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b66396-ef6b-4489-81b7-93e67bf9eff2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:13.167325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12aea9a-5572-498b-8d83-14818128831a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:13.168619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12aea9a-5572-498b-8d83-14818128831a service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:13.168907 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12aea9a-5572-498b-8d83-14818128831a service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:13.169190 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12aea9a-5572-498b-8d83-14818128831a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:13.244226 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12aea9a-5572-498b-8d83-14818128831a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:13.248061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12aea9a-5572-498b-8d83-14818128831a service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:13.269604 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1235/2452] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:13 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:18.216910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2750dc-9f54-4d24-acb4-df97d8aa17a0 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:18.218444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2e2750dc-9f54-4d24-acb4-df97d8aa17a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:18.237782 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2750dc-9f54-4d24-acb4-df97d8aa17a0 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:18.244254 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a380a80e-09eb-4c50-97ff-4328879a220d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:18.245092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380a80e-09eb-4c50-97ff-4328879a220d service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:18.245414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380a80e-09eb-4c50-97ff-4328879a220d service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:18.245994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a380a80e-09eb-4c50-97ff-4328879a220d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:18.247736 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a380a80e-09eb-4c50-97ff-4328879a220d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:18.306565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:18.320187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380a80e-09eb-4c50-97ff-4328879a220d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:18.322008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a380a80e-09eb-4c50-97ff-4328879a220d service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:18.333196 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1218/2453] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:33:23.255925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:INSERT=3,keystone:SELECT=73 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:24.729392 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12aea9a-5572-498b-8d83-14818128831a service nova] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:24.741276 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12aea9a-5572-498b-8d83-14818128831a service nova] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:24.747842 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8006e8-5f05-40e1-9aa8-4aed7ebf0a7b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:24.748538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8006e8-5f05-40e1-9aa8-4aed7ebf0a7b service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:24.748825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8006e8-5f05-40e1-9aa8-4aed7ebf0a7b service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:24.749104 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8006e8-5f05-40e1-9aa8-4aed7ebf0a7b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:24.751093 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4e8006e8-5f05-40e1-9aa8-4aed7ebf0a7b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:24.818461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8006e8-5f05-40e1-9aa8-4aed7ebf0a7b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:24.820080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8006e8-5f05-40e1-9aa8-4aed7ebf0a7b service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:24.831013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1236/2454] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:24 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:28.328645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:34.826065 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=28 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:38.267300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a33ffb-470e-4bf9-941f-5448ef879437 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:38.267885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a33ffb-470e-4bf9-941f-5448ef879437 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:38.267885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a33ffb-470e-4bf9-941f-5448ef879437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:38.284812 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b1a33ffb-470e-4bf9-941f-5448ef879437 None None] MFA Rules not processed for user `8aa0b8a0b7ff4eb0b13fb985dc0d710a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:38.286061 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b1a33ffb-470e-4bf9-941f-5448ef879437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:38.293022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1219/2455] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:33:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:38.623940 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8006e8-5f05-40e1-9aa8-4aed7ebf0a7b service nova] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:38.681287 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8006e8-5f05-40e1-9aa8-4aed7ebf0a7b service nova] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:38.686859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fd1fc-286f-4f9a-a227-1189c3733203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:38.686859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fd1fc-286f-4f9a-a227-1189c3733203 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:38.686859 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fd1fc-286f-4f9a-a227-1189c3733203 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:38.687234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5fd1fc-286f-4f9a-a227-1189c3733203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:38.688933 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8a5fd1fc-286f-4f9a-a227-1189c3733203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:38.704824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fd1fc-286f-4f9a-a227-1189c3733203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:38.708124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5fd1fc-286f-4f9a-a227-1189c3733203 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:38.723419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1237/2456] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:33:40.312987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a33ffb-470e-4bf9-941f-5448ef879437 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:40.314513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b1a33ffb-470e-4bf9-941f-5448ef879437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:40.388599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a33ffb-470e-4bf9-941f-5448ef879437 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:40.394281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b760c7-3fad-4967-b6c7-f4c07dcfff51 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e97119806774c2482a9886b11ab6f70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:40.395123 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b760c7-3fad-4967-b6c7-f4c07dcfff51 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:40.395539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b760c7-3fad-4967-b6c7-f4c07dcfff51 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:40.395914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b760c7-3fad-4967-b6c7-f4c07dcfff51 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:40.399924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-36b760c7-3fad-4967-b6c7-f4c07dcfff51 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:40.411187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b760c7-3fad-4967-b6c7-f4c07dcfff51 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:40.412811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b760c7-3fad-4967-b6c7-f4c07dcfff51 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:40.424955 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1220/2457] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:45.928977 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fd1fc-286f-4f9a-a227-1189c3733203 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:45.952351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5fd1fc-286f-4f9a-a227-1189c3733203 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:45.959196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97443cfd-a2fb-4f98-8109-5ca22e29fa94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:45.959867 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97443cfd-a2fb-4f98-8109-5ca22e29fa94 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:45.960150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97443cfd-a2fb-4f98-8109-5ca22e29fa94 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:45.960424 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97443cfd-a2fb-4f98-8109-5ca22e29fa94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:45.962318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-97443cfd-a2fb-4f98-8109-5ca22e29fa94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:45.978289 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97443cfd-a2fb-4f98-8109-5ca22e29fa94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:45.980862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97443cfd-a2fb-4f98-8109-5ca22e29fa94 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:45.996090 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1238/2458] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:50.420704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=35 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:33:53.176849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b760c7-3fad-4967-b6c7-f4c07dcfff51 service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:53.192039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b760c7-3fad-4967-b6c7-f4c07dcfff51 service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:53.198913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4ee32-dc30-443f-9327-201b8667863c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:53.199865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4ee32-dc30-443f-9327-201b8667863c service nova] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:53.200240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4ee32-dc30-443f-9327-201b8667863c service nova] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:53.200584 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4ee32-dc30-443f-9327-201b8667863c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:53.203497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6dc4ee32-dc30-443f-9327-201b8667863c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:53.216965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4ee32-dc30-443f-9327-201b8667863c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:53.218796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4ee32-dc30-443f-9327-201b8667863c service nova] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:53.245762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1221/2459] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:33:54.470409 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95a3d9-12de-4cd5-8f0d-e806900defb5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:54.470920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95a3d9-12de-4cd5-8f0d-e806900defb5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:54.470920 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95a3d9-12de-4cd5-8f0d-e806900defb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:54.489318 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4b95a3d9-12de-4cd5-8f0d-e806900defb5 None None] MFA Rules not processed for user `8f448776d1074e22bfe6cfbe05e01db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:33:54.490588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4b95a3d9-12de-4cd5-8f0d-e806900defb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:33:54.497667 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1239/2460] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:33:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:33:56.793742 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4ee32-dc30-443f-9327-201b8667863c service nova] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:33:56.803141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4ee32-dc30-443f-9327-201b8667863c service nova] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:33:56.806696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380ab775-3593-4de2-bfea-60217cedb01e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:33:56.807327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ab775-3593-4de2-bfea-60217cedb01e service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:33:56.807611 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ab775-3593-4de2-bfea-60217cedb01e service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:33:56.807885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380ab775-3593-4de2-bfea-60217cedb01e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:33:56.817922 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ab775-3593-4de2-bfea-60217cedb01e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:33:56.819340 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380ab775-3593-4de2-bfea-60217cedb01e service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:33:56.829724 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1222/2461] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:33:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:00.887910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95a3d9-12de-4cd5-8f0d-e806900defb5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:00.906339 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95a3d9-12de-4cd5-8f0d-e806900defb5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:00.913470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f02c1d1-784c-4bcf-b08e-84b9e9777287 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1cc0050f14664630a36814ca670f124f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:00.914146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02c1d1-784c-4bcf-b08e-84b9e9777287 service nova] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:00.914437 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02c1d1-784c-4bcf-b08e-84b9e9777287 service nova] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:00.914718 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f02c1d1-784c-4bcf-b08e-84b9e9777287 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:00.916749 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-9f02c1d1-784c-4bcf-b08e-84b9e9777287 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:00.929851 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02c1d1-784c-4bcf-b08e-84b9e9777287 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:00.931461 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f02c1d1-784c-4bcf-b08e-84b9e9777287 service nova] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:00.942925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1240/2462] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:34:06.824732 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=12 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:10.937930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=41 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:23.504422 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3750884-88bf-4d46-b1a3-cc5a83c9a3f3 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:23.504965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3750884-88bf-4d46-b1a3-cc5a83c9a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:23.504965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3750884-88bf-4d46-b1a3-cc5a83c9a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:23.523621 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f3750884-88bf-4d46-b1a3-cc5a83c9a3f3 None None] MFA Rules not processed for user `f8de4128718d4837af77d427973d30be`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:23.524846 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f3750884-88bf-4d46-b1a3-cc5a83c9a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:23.581141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1223/2463] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:34:23 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:33.574011 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=25 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:34:52.029904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad09a6c-7613-4be5-8aa9-5956cffbb9a8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:52.030484 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad09a6c-7613-4be5-8aa9-5956cffbb9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:52.030590 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad09a6c-7613-4be5-8aa9-5956cffbb9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:52.048799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4ad09a6c-7613-4be5-8aa9-5956cffbb9a8 None None] MFA Rules not processed for user `c4b710bbdf0c4d58b4f3ee7ef7410c0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:34:52.050050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4ad09a6c-7613-4be5-8aa9-5956cffbb9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:52.059779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1241/2464] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:34:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:34:52.407582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3750884-88bf-4d46-b1a3-cc5a83c9a3f3 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:34:52.423871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3750884-88bf-4d46-b1a3-cc5a83c9a3f3 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:34:52.428569 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:34:52.429227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:34:52.429547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:34:52.429828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:34:52.431706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:34:52.442331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:34:52.443777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:34:52.452004 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1224/2465] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:34:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:02.043761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:02.448326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=6 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:22.507905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad09a6c-7613-4be5-8aa9-5956cffbb9a8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:22.528367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad09a6c-7613-4be5-8aa9-5956cffbb9a8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:22.531910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:22.532673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:22.532956 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:22.533237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:22.535080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:22.659960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:22.661430 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:22.680647 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1242/2466] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:25.978987 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:25.981830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:26.041486 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:26.042928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:26.055400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633acd5e-dd87-43a1-8235-64d1d28954b9 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.059930 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.061134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.061451 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.061732 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] PATH_INFO: `/v3/users/8f448776d1074e22bfe6cfbe05e01db7` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.062971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f448776d1074e22bfe6cfbe05e01db7)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.065548 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:26.069624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63fef9-96c4-429a-8ef8-2cdee97fe4f2 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.074710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9615d0a-5d25-42e5-b322-126b62bb66d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e97119806774c2482a9886b11ab6f70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.075425 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9615d0a-5d25-42e5-b322-126b62bb66d0 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.075777 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9615d0a-5d25-42e5-b322-126b62bb66d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.076147 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9615d0a-5d25-42e5-b322-126b62bb66d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.078045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e9615d0a-5d25-42e5-b322-126b62bb66d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:26.090974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9615d0a-5d25-42e5-b322-126b62bb66d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.092914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9615d0a-5d25-42e5-b322-126b62bb66d0 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:26.104404 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1225/2467] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:26.116479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f448776d1074e22bfe6cfbe05e01db7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.123990 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f448776d1074e22bfe6cfbe05e01db7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.124664 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f448776d1074e22bfe6cfbe05e01db7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.132906 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f448776d1074e22bfe6cfbe05e01db7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.147115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-da471ef5-96ef-4755-b0a9-a2e2858d0f28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f448776d1074e22bfe6cfbe05e01db7'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.148684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:26.154441 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1243/2468] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:35:26 2022] DELETE /identity/v3/users/8f448776d1074e22bfe6cfbe05e01db7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:35:26.330880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9615d0a-5d25-42e5-b322-126b62bb66d0 service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:26.342524 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9615d0a-5d25-42e5-b322-126b62bb66d0 service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.400849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.401740 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.402021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.402321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] PATH_INFO: `/v3/projects/2c27fe465e6743e19feec424e6e9f3d1` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.403470 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c27fe465e6743e19feec424e6e9f3d1)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.405368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:26.451838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c27fe465e6743e19feec424e6e9f3d1', 'request_id': 'req-ec02a1fe-d81d-4b12-bbfc-da18db319439'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.460256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c27fe465e6743e19feec424e6e9f3d1', 'request_id': 'req-ec02a1fe-d81d-4b12-bbfc-da18db319439'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.471606 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c27fe465e6743e19feec424e6e9f3d1', 'request_id': 'req-ec02a1fe-d81d-4b12-bbfc-da18db319439'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.485350 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] The token cache is being invalidate because project 2c27fe465e6743e19feec424e6e9f3d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:35:26.485749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.487065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-ec02a1fe-d81d-4b12-bbfc-da18db319439 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:26.488693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1226/2469] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:35:26 2022] DELETE /identity/v3/projects/2c27fe465e6743e19feec424e6e9f3d1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:26.493831 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cd6d8-9eb1-4039-a90f-063c8ff14934 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.494117 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cd6d8-9eb1-4039-a90f-063c8ff14934 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.494394 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3cd6d8-9eb1-4039-a90f-063c8ff14934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.557696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-3c3cd6d8-9eb1-4039-a90f-063c8ff14934 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:26.558928 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-3c3cd6d8-9eb1-4039-a90f-063c8ff14934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:26.637925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1244/2470] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:26.647599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951684b-dd77-4b9a-bdbd-a9c506cb3c81 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.647925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951684b-dd77-4b9a-bdbd-a9c506cb3c81 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.648179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5951684b-dd77-4b9a-bdbd-a9c506cb3c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.670199 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5951684b-dd77-4b9a-bdbd-a9c506cb3c81 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:26.671441 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5951684b-dd77-4b9a-bdbd-a9c506cb3c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:26.679938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1227/2471] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:35:26.684250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cd6d8-9eb1-4039-a90f-063c8ff14934 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:26.695422 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3cd6d8-9eb1-4039-a90f-063c8ff14934 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.707954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bfda4-1096-4ccf-a423-ccfdc7289a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.708707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bfda4-1096-4ccf-a423-ccfdc7289a12 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.708996 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bfda4-1096-4ccf-a423-ccfdc7289a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.709300 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432bfda4-1096-4ccf-a423-ccfdc7289a12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.712089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bfda4-1096-4ccf-a423-ccfdc7289a12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:35:26.712089 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bfda4-1096-4ccf-a423-ccfdc7289a12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.712400 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432bfda4-1096-4ccf-a423-ccfdc7289a12 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:26.725938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1245/2472] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:35:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:35:26.730285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951684b-dd77-4b9a-bdbd-a9c506cb3c81 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:26.740926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5951684b-dd77-4b9a-bdbd-a9c506cb3c81 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.745683 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0500d-0736-477f-b11d-06ee151d2de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.746343 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0500d-0736-477f-b11d-06ee151d2de3 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.746632 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0500d-0736-477f-b11d-06ee151d2de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.746913 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c0500d-0736-477f-b11d-06ee151d2de3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.747635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0500d-0736-477f-b11d-06ee151d2de3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.749599 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c0500d-0736-477f-b11d-06ee151d2de3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:26.770874 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:26.778741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1228/2473] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:35:26 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:26.782891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bfda4-1096-4ccf-a423-ccfdc7289a12 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:26.794295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432bfda4-1096-4ccf-a423-ccfdc7289a12 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.851766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f712e-d615-4c1d-8e29-653bb54eaf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.852449 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f712e-d615-4c1d-8e29-653bb54eaf1c admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.852734 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f712e-d615-4c1d-8e29-653bb54eaf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.853010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1f712e-d615-4c1d-8e29-653bb54eaf1c admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.853781 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f712e-d615-4c1d-8e29-653bb54eaf1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.855714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1f712e-d615-4c1d-8e29-653bb54eaf1c admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:26.879871 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1246/2474] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:35:26 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:35:26.881088 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0500d-0736-477f-b11d-06ee151d2de3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:26.891696 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c0500d-0736-477f-b11d-06ee151d2de3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.897042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fccb80-5774-4d3c-8032-e8c564f0fe62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.897753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fccb80-5774-4d3c-8032-e8c564f0fe62 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.898041 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fccb80-5774-4d3c-8032-e8c564f0fe62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.898321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fccb80-5774-4d3c-8032-e8c564f0fe62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.899060 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fccb80-5774-4d3c-8032-e8c564f0fe62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.900679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fccb80-5774-4d3c-8032-e8c564f0fe62 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:26.912820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1229/2475] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:35:26.916862 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f712e-d615-4c1d-8e29-653bb54eaf1c admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:26.927764 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1f712e-d615-4c1d-8e29-653bb54eaf1c admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.932374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfcd243-bb0d-4b44-86c8-c5c4dfb38b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.933032 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcd243-bb0d-4b44-86c8-c5c4dfb38b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.933315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcd243-bb0d-4b44-86c8-c5c4dfb38b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.933621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfcd243-bb0d-4b44-86c8-c5c4dfb38b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.934351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcd243-bb0d-4b44-86c8-c5c4dfb38b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.935950 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfcd243-bb0d-4b44-86c8-c5c4dfb38b78 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:26.946931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1247/2476] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:26.951243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fccb80-5774-4d3c-8032-e8c564f0fe62 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:26.961332 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fccb80-5774-4d3c-8032-e8c564f0fe62 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:26.965984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836fca3-9fcc-445e-ade1-b00313c63066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:26.966875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836fca3-9fcc-445e-ade1-b00313c63066 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:26.967168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836fca3-9fcc-445e-ade1-b00313c63066 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:26.967451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d836fca3-9fcc-445e-ade1-b00313c63066 admin admin] PATH_INFO: `/v3/projects/4b9cb5a8bade4c2b9d9abf8e327b4531/users/0899eec66fcc4c3c8f213a08aa094b3a/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:26.984120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836fca3-9fcc-445e-ade1-b00313c63066 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9cb5a8bade4c2b9d9abf8e327b4531, user_id=0899eec66fcc4c3c8f213a08aa094b3a, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:26.987627 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d836fca3-9fcc-445e-ade1-b00313c63066 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:27.000785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1230/2477] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:26 2022] PUT /identity/v3/projects/4b9cb5a8bade4c2b9d9abf8e327b4531/users/0899eec66fcc4c3c8f213a08aa094b3a/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:27.011619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55ea51-1727-48bd-bdb4-8b015257f91e None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:27.011923 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55ea51-1727-48bd-bdb4-8b015257f91e None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:27.012210 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a55ea51-1727-48bd-bdb4-8b015257f91e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:27.041516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-6a55ea51-1727-48bd-bdb4-8b015257f91e None None] MFA Rules not processed for user `0899eec66fcc4c3c8f213a08aa094b3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:27.042752 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-6a55ea51-1727-48bd-bdb4-8b015257f91e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:27.063260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:27.109874 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1248/2478] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:27 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:27.120663 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836fca3-9fcc-445e-ade1-b00313c63066 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:27.122375 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d836fca3-9fcc-445e-ade1-b00313c63066 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:27.182381 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d836fca3-9fcc-445e-ade1-b00313c63066 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:27.186135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00508504-4b23-4372-ad5f-2f13a3f8fabe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:27.186788 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508504-4b23-4372-ad5f-2f13a3f8fabe service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:27.187072 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508504-4b23-4372-ad5f-2f13a3f8fabe service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:27.187379 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00508504-4b23-4372-ad5f-2f13a3f8fabe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:27.197209 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00508504-4b23-4372-ad5f-2f13a3f8fabe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:27.198684 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00508504-4b23-4372-ad5f-2f13a3f8fabe service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:27.208313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1231/2479] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:27 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:35.576477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55ea51-1727-48bd-bdb4-8b015257f91e None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:35.638644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a55ea51-1727-48bd-bdb4-8b015257f91e None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:35.643390 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:35.644258 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:35.644539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:35.644809 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] PATH_INFO: `/v3/users/0899eec66fcc4c3c8f213a08aa094b3a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:35.646325 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0899eec66fcc4c3c8f213a08aa094b3a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:35.648740 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:35.704130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0899eec66fcc4c3c8f213a08aa094b3a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:35.711863 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0899eec66fcc4c3c8f213a08aa094b3a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:35.712565 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0899eec66fcc4c3c8f213a08aa094b3a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:35.720268 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0899eec66fcc4c3c8f213a08aa094b3a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:35.733535 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0899eec66fcc4c3c8f213a08aa094b3a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:35.740791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1249/2480] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:35:35 2022] DELETE /identity/v3/users/0899eec66fcc4c3c8f213a08aa094b3a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:35.745694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaa596-f70a-4ab0-a605-1ba54ad8d1b7 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:35.746206 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaa596-f70a-4ab0-a605-1ba54ad8d1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:35.746309 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aaa596-f70a-4ab0-a605-1ba54ad8d1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:35.763326 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-e2aaa596-f70a-4ab0-a605-1ba54ad8d1b7 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:35.764548 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e2aaa596-f70a-4ab0-a605-1ba54ad8d1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:35.825008 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1232/2481] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:35.834480 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:35.835887 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:35.882684 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:35.917555 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de36f2-20d1-41fa-a346-78451bfb987b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:35.921225 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de38d8b-d4d4-481b-971b-c7c315ac8b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:35.921899 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de38d8b-d4d4-481b-971b-c7c315ac8b44 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:35.922185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de38d8b-d4d4-481b-971b-c7c315ac8b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:35.922464 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de38d8b-d4d4-481b-971b-c7c315ac8b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:35.932489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de38d8b-d4d4-481b-971b-c7c315ac8b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:35.933954 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de38d8b-d4d4-481b-971b-c7c315ac8b44 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:35.944142 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1250/2482] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:35:36.242412 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaa596-f70a-4ab0-a605-1ba54ad8d1b7 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:36.244705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e2aaa596-f70a-4ab0-a605-1ba54ad8d1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:36.265054 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aaa596-f70a-4ab0-a605-1ba54ad8d1b7 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:36.273324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:36.274319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.274759 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.275242 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] PATH_INFO: `/v3/projects/4b9cb5a8bade4c2b9d9abf8e327b4531` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.276932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9cb5a8bade4c2b9d9abf8e327b4531)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:36.279787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:36.341640 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9cb5a8bade4c2b9d9abf8e327b4531', 'request_id': 'req-f82ed350-8282-47e1-9131-917755f9c84f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:36.348587 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9cb5a8bade4c2b9d9abf8e327b4531', 'request_id': 'req-f82ed350-8282-47e1-9131-917755f9c84f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:36.353056 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:36.359282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9cb5a8bade4c2b9d9abf8e327b4531', 'request_id': 'req-f82ed350-8282-47e1-9131-917755f9c84f'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:36.372414 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] The token cache is being invalidate because project 4b9cb5a8bade4c2b9d9abf8e327b4531 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:35:36.372706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:36.373468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f82ed350-8282-47e1-9131-917755f9c84f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:36.374957 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1233/2483] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:35:36 2022] DELETE /identity/v3/projects/4b9cb5a8bade4c2b9d9abf8e327b4531 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:36.380005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6801f-24ad-43c2-8465-403eb3aab2a4 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.380281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6801f-24ad-43c2-8465-403eb3aab2a4 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.380569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f6801f-24ad-43c2-8465-403eb3aab2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.399824 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-70f6801f-24ad-43c2-8465-403eb3aab2a4 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:36.401040 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-70f6801f-24ad-43c2-8465-403eb3aab2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:36.463514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1251/2484] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:36.471924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccf0d9-669c-467b-801e-7a1aa2efe384 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.472486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccf0d9-669c-467b-801e-7a1aa2efe384 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.472770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccf0d9-669c-467b-801e-7a1aa2efe384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.496453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7cccf0d9-669c-467b-801e-7a1aa2efe384 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:36.497711 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7cccf0d9-669c-467b-801e-7a1aa2efe384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:36.505923 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1234/2485] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:36.510255 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6801f-24ad-43c2-8465-403eb3aab2a4 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:36.520314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f6801f-24ad-43c2-8465-403eb3aab2a4 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:36.524899 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e22dfa-d955-4e17-89c8-5437454a2f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:36.525592 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22dfa-d955-4e17-89c8-5437454a2f81 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.525900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22dfa-d955-4e17-89c8-5437454a2f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.526158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e22dfa-d955-4e17-89c8-5437454a2f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.526888 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e22dfa-d955-4e17-89c8-5437454a2f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:35:36.527196 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e22dfa-d955-4e17-89c8-5437454a2f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:36.528791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e22dfa-d955-4e17-89c8-5437454a2f81 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:36.542720 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1252/2486] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:35:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:35:36.547654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccf0d9-669c-467b-801e-7a1aa2efe384 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:36.564259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccf0d9-669c-467b-801e-7a1aa2efe384 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:36.572184 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced31ca6-d4c8-4022-90cd-63d2882dcea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:36.573311 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced31ca6-d4c8-4022-90cd-63d2882dcea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.573835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced31ca6-d4c8-4022-90cd-63d2882dcea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.574303 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced31ca6-d4c8-4022-90cd-63d2882dcea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.575410 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced31ca6-d4c8-4022-90cd-63d2882dcea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:36.578960 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced31ca6-d4c8-4022-90cd-63d2882dcea8 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:36.611185 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1235/2487] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:35:36 2022] POST /identity/v3/projects => generated 357 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:36.616050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e22dfa-d955-4e17-89c8-5437454a2f81 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:36.626619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e22dfa-d955-4e17-89c8-5437454a2f81 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:36.699379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343aa4fc-6cd1-46c9-9b22-30ca6361c5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:36.699379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aa4fc-6cd1-46c9-9b22-30ca6361c5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.699379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aa4fc-6cd1-46c9-9b22-30ca6361c5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.699379 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343aa4fc-6cd1-46c9-9b22-30ca6361c5f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.699934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343aa4fc-6cd1-46c9-9b22-30ca6361c5f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:36.699934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343aa4fc-6cd1-46c9-9b22-30ca6361c5f1 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:36.719378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1253/2488] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:35:36 2022] POST /identity/v3/users => generated 279 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:35:36.724188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced31ca6-d4c8-4022-90cd-63d2882dcea8 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:36.734625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced31ca6-d4c8-4022-90cd-63d2882dcea8 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:36.739830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065d944-8953-44bf-966b-315b46d31de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:36.740496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065d944-8953-44bf-966b-315b46d31de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.740785 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065d944-8953-44bf-966b-315b46d31de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.741067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065d944-8953-44bf-966b-315b46d31de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.741844 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065d944-8953-44bf-966b-315b46d31de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:36.743449 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065d944-8953-44bf-966b-315b46d31de6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:36.754364 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1236/2489] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:36.758420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343aa4fc-6cd1-46c9-9b22-30ca6361c5f1 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:36.768641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343aa4fc-6cd1-46c9-9b22-30ca6361c5f1 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:36.773220 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9d30c3-5fb4-4277-9db6-345c9de2ca64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:36.773941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d30c3-5fb4-4277-9db6-345c9de2ca64 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.774229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d30c3-5fb4-4277-9db6-345c9de2ca64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.774497 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9d30c3-5fb4-4277-9db6-345c9de2ca64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.775254 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d30c3-5fb4-4277-9db6-345c9de2ca64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:36.776828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9d30c3-5fb4-4277-9db6-345c9de2ca64 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:36.787837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1254/2490] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:36.791928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065d944-8953-44bf-966b-315b46d31de6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:36.802917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065d944-8953-44bf-966b-315b46d31de6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:36.807608 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aca281-5b3a-404c-9680-71644596d764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:36.808496 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aca281-5b3a-404c-9680-71644596d764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.808786 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aca281-5b3a-404c-9680-71644596d764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.809067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aca281-5b3a-404c-9680-71644596d764 admin admin] PATH_INFO: `/v3/projects/ce03056d605b4e089244a11155ecbc0f/users/fb336b10f573432daf5c3a5bf177e70d/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.824556 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aca281-5b3a-404c-9680-71644596d764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce03056d605b4e089244a11155ecbc0f, user_id=fb336b10f573432daf5c3a5bf177e70d, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:36.827279 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aca281-5b3a-404c-9680-71644596d764 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:36.843406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1237/2491] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:36 2022] PUT /identity/v3/projects/ce03056d605b4e089244a11155ecbc0f/users/fb336b10f573432daf5c3a5bf177e70d/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:36.858678 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c2a8-b332-4764-936a-432dc6018d81 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:36.858971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c2a8-b332-4764-936a-432dc6018d81 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:36.859275 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013c2a8-b332-4764-936a-432dc6018d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:36.886330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-2013c2a8-b332-4764-936a-432dc6018d81 None None] MFA Rules not processed for user `fb336b10f573432daf5c3a5bf177e70d`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:36.887458 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2013c2a8-b332-4764-936a-432dc6018d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:36.906352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:36.954415 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1255/2492] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:36 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:36.959646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aca281-5b3a-404c-9680-71644596d764 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:37.024349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aca281-5b3a-404c-9680-71644596d764 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:37.029798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b187f8-db8c-4735-9958-1157678ca75a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:37.030466 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b187f8-db8c-4735-9958-1157678ca75a admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:37.030750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b187f8-db8c-4735-9958-1157678ca75a admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:37.031026 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b187f8-db8c-4735-9958-1157678ca75a admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:37.031729 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b187f8-db8c-4735-9958-1157678ca75a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:37.034673 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b187f8-db8c-4735-9958-1157678ca75a admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:37.062089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1238/2493] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:35:36 2022] POST /identity/v3/projects => generated 355 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:37.067199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013c2a8-b332-4764-936a-432dc6018d81 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:37.077990 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013c2a8-b332-4764-936a-432dc6018d81 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:37.135471 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191a4b1-eb9a-487f-a630-0b761cc1b6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:37.136144 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191a4b1-eb9a-487f-a630-0b761cc1b6e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:37.136483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191a4b1-eb9a-487f-a630-0b761cc1b6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:37.136707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0191a4b1-eb9a-487f-a630-0b761cc1b6e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:37.137469 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0191a4b1-eb9a-487f-a630-0b761cc1b6e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:37.139404 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0191a4b1-eb9a-487f-a630-0b761cc1b6e9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:37.160293 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1256/2494] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:35:37 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:37.164472 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b187f8-db8c-4735-9958-1157678ca75a admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:37.179664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b187f8-db8c-4735-9958-1157678ca75a admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:37.184426 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe9e77-0771-4b49-b383-04b0277a60f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:37.185146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe9e77-0771-4b49-b383-04b0277a60f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:37.185463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe9e77-0771-4b49-b383-04b0277a60f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:37.185737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfe9e77-0771-4b49-b383-04b0277a60f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:37.186468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe9e77-0771-4b49-b383-04b0277a60f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:37.188099 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfe9e77-0771-4b49-b383-04b0277a60f2 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:37.200146 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1239/2495] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:35:37.204384 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191a4b1-eb9a-487f-a630-0b761cc1b6e9 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:37.215053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0191a4b1-eb9a-487f-a630-0b761cc1b6e9 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:37.219633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad47e1-4b76-4ff7-a084-ab8ef05268a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:37.220513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad47e1-4b76-4ff7-a084-ab8ef05268a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:37.220837 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad47e1-4b76-4ff7-a084-ab8ef05268a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:37.221462 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bad47e1-4b76-4ff7-a084-ab8ef05268a7 admin admin] PATH_INFO: `/v3/projects/3a872378b9924bbbaed90e8c851018b1/users/ce5f2d59dad8458f92eabc960fc76bd4/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:37.236562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad47e1-4b76-4ff7-a084-ab8ef05268a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a872378b9924bbbaed90e8c851018b1, user_id=ce5f2d59dad8458f92eabc960fc76bd4, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:37.239468 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bad47e1-4b76-4ff7-a084-ab8ef05268a7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:37.254902 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1257/2496] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:37 2022] PUT /identity/v3/projects/3a872378b9924bbbaed90e8c851018b1/users/ce5f2d59dad8458f92eabc960fc76bd4/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:37.259080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe9e77-0771-4b49-b383-04b0277a60f2 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:37.270452 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfe9e77-0771-4b49-b383-04b0277a60f2 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:37.326534 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae07687-0695-468c-b5fe-ce54dc74a8c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:37.327211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae07687-0695-468c-b5fe-ce54dc74a8c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:37.327497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae07687-0695-468c-b5fe-ce54dc74a8c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:37.327783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae07687-0695-468c-b5fe-ce54dc74a8c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:37.328513 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae07687-0695-468c-b5fe-ce54dc74a8c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:37.330732 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae07687-0695-468c-b5fe-ce54dc74a8c4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:37.332917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:37.342132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1240/2497] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:37.346605 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad47e1-4b76-4ff7-a084-ab8ef05268a7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:37.356757 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bad47e1-4b76-4ff7-a084-ab8ef05268a7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:37.362251 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db740970-4af9-489b-9914-278136514f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:37.363201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db740970-4af9-489b-9914-278136514f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:37.363493 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db740970-4af9-489b-9914-278136514f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:37.363798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db740970-4af9-489b-9914-278136514f84 admin admin] PATH_INFO: `/v3/projects/3a872378b9924bbbaed90e8c851018b1/users/ce5f2d59dad8458f92eabc960fc76bd4/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:37.365849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db740970-4af9-489b-9914-278136514f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a872378b9924bbbaed90e8c851018b1, user_id=ce5f2d59dad8458f92eabc960fc76bd4, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:37.368514 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db740970-4af9-489b-9914-278136514f84 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:37.380533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1258/2498] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:37 2022] PUT /identity/v3/projects/3a872378b9924bbbaed90e8c851018b1/users/ce5f2d59dad8458f92eabc960fc76bd4/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:37.390732 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19185e8-59ce-4762-94ce-7d2070dc5640 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:37.390961 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19185e8-59ce-4762-94ce-7d2070dc5640 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:37.391247 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19185e8-59ce-4762-94ce-7d2070dc5640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:37.420090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-f19185e8-59ce-4762-94ce-7d2070dc5640 None None] MFA Rules not processed for user `ce5f2d59dad8458f92eabc960fc76bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:37.421487 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f19185e8-59ce-4762-94ce-7d2070dc5640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:37.496460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1241/2499] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:37 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:37.508014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db740970-4af9-489b-9914-278136514f84 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:37.509766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-db740970-4af9-489b-9914-278136514f84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:37.587223 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db740970-4af9-489b-9914-278136514f84 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:37.590890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f61d3-560d-48ee-9718-999310cfe405 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:37.591538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f61d3-560d-48ee-9718-999310cfe405 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:37.591828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f61d3-560d-48ee-9718-999310cfe405 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:37.592166 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f61d3-560d-48ee-9718-999310cfe405 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:37.599147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:37.601354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f61d3-560d-48ee-9718-999310cfe405 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:37.603338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f61d3-560d-48ee-9718-999310cfe405 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:37.614068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1259/2500] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:37 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:37.683919 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19185e8-59ce-4762-94ce-7d2070dc5640 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:37.694176 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19185e8-59ce-4762-94ce-7d2070dc5640 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:37.697798 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c2e71-a072-4e78-9440-a8658b79d456 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:37.698448 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c2e71-a072-4e78-9440-a8658b79d456 service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:37.698756 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c2e71-a072-4e78-9440-a8658b79d456 service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:37.699042 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c2e71-a072-4e78-9440-a8658b79d456 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:37.757361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c2e71-a072-4e78-9440-a8658b79d456 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:37.758871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c2e71-a072-4e78-9440-a8658b79d456 service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:37.767865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1242/2501] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:37 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:47.609418 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=2 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:47.763764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=63 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:52.694294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f61d3-560d-48ee-9718-999310cfe405 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:52.757626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f61d3-560d-48ee-9718-999310cfe405 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:52.762005 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:52.762886 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:52.763176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:52.763453 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] PATH_INFO: `/v3/users/fb336b10f573432daf5c3a5bf177e70d` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:52.764652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb336b10f573432daf5c3a5bf177e70d)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:52.766643 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:52.814444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb336b10f573432daf5c3a5bf177e70d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:52.821697 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb336b10f573432daf5c3a5bf177e70d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:52.822401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb336b10f573432daf5c3a5bf177e70d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:52.829975 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb336b10f573432daf5c3a5bf177e70d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:52.842582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb336b10f573432daf5c3a5bf177e70d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:52.849550 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1260/2502] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:35:52 2022] DELETE /identity/v3/users/fb336b10f573432daf5c3a5bf177e70d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:35:52.854259 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9b704-b413-4701-a0e4-593b5d51a83a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:52.854772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9b704-b413-4701-a0e4-593b5d51a83a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:52.854871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c9b704-b413-4701-a0e4-593b5d51a83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:52.874783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-54c9b704-b413-4701-a0e4-593b5d51a83a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:52.875979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-54c9b704-b413-4701-a0e4-593b5d51a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:52.942218 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1243/2503] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:52.952080 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:52.954136 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:53.013778 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2475ad0b-b487-4e6f-b016-4ac98b6b6123 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:53.017539 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f6f4-67ba-488c-b869-12feda665d13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:53.018782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f6f4-67ba-488c-b869-12feda665d13 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:53.019066 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f6f4-67ba-488c-b869-12feda665d13 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:53.019342 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6f6f4-67ba-488c-b869-12feda665d13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:53.030485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6f6f4-67ba-488c-b869-12feda665d13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:53.031938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d6f6f4-67ba-488c-b869-12feda665d13 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:53.042517 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1261/2504] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:53.342664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9b704-b413-4701-a0e4-593b5d51a83a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:53.344403 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-54c9b704-b413-4701-a0e4-593b5d51a83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:53.359789 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c9b704-b413-4701-a0e4-593b5d51a83a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:53.371313 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:53.372236 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:53.372522 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:53.372820 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] PATH_INFO: `/v3/projects/ce03056d605b4e089244a11155ecbc0f` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:53.375303 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce03056d605b4e089244a11155ecbc0f)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:53.378928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:53.434637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce03056d605b4e089244a11155ecbc0f', 'request_id': 'req-f3fce293-0c8f-40c9-970f-add041aba800'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.443175 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce03056d605b4e089244a11155ecbc0f', 'request_id': 'req-f3fce293-0c8f-40c9-970f-add041aba800'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.451387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce03056d605b4e089244a11155ecbc0f', 'request_id': 'req-f3fce293-0c8f-40c9-970f-add041aba800'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.462895 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] The token cache is being invalidate because project ce03056d605b4e089244a11155ecbc0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:35:53.463188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.463855 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.465446 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1244/2505] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:35:53 2022] DELETE /identity/v3/projects/ce03056d605b4e089244a11155ecbc0f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:53.469659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f6f4-67ba-488c-b869-12feda665d13 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:53.480485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6f6f4-67ba-488c-b869-12feda665d13 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:53.500172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:53.543913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:53.546606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:53.546606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:53.546606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] PATH_INFO: `/v3/users/ce5f2d59dad8458f92eabc960fc76bd4` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:53.546606 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce5f2d59dad8458f92eabc960fc76bd4)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:53.548516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:53.605890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce5f2d59dad8458f92eabc960fc76bd4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.614370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce5f2d59dad8458f92eabc960fc76bd4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.615120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce5f2d59dad8458f92eabc960fc76bd4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.624341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce5f2d59dad8458f92eabc960fc76bd4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.637723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-a0f92266-037d-4f88-95aa-143f7e828500 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce5f2d59dad8458f92eabc960fc76bd4'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:53.648216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1262/2506] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:35:53 2022] DELETE /identity/v3/users/ce5f2d59dad8458f92eabc960fc76bd4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:53.927380 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:53.931297 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:53.945668 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fce293-0c8f-40c9-970f-add041aba800 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.011753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.012637 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.012914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.013187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] PATH_INFO: `/v3/projects/3a872378b9924bbbaed90e8c851018b1` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.014419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a872378b9924bbbaed90e8c851018b1)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.016335 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.050131 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:54.069249 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a872378b9924bbbaed90e8c851018b1', 'request_id': 'req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:54.077200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a872378b9924bbbaed90e8c851018b1', 'request_id': 'req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:54.086859 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a872378b9924bbbaed90e8c851018b1', 'request_id': 'req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:54.099811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] The token cache is being invalidate because project 3a872378b9924bbbaed90e8c851018b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:35:54.100104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:54.100776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d4f40e3d-adfc-4d32-a2a1-fd05049f015c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:54.102419 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1245/2507] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:35:53 2022] DELETE /identity/v3/projects/3a872378b9924bbbaed90e8c851018b1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:54.107192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cf43b-129a-47d0-9004-def0b5d6c447 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.107666 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cf43b-129a-47d0-9004-def0b5d6c447 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.107776 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35cf43b-129a-47d0-9004-def0b5d6c447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.126618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-e35cf43b-129a-47d0-9004-def0b5d6c447 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:54.127910 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e35cf43b-129a-47d0-9004-def0b5d6c447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:54.189756 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1263/2508] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:54.199700 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf607438-a59c-4e9c-849e-55664398579a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.199984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf607438-a59c-4e9c-849e-55664398579a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.200282 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf607438-a59c-4e9c-849e-55664398579a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.219938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cf607438-a59c-4e9c-849e-55664398579a None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:54.221216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cf607438-a59c-4e9c-849e-55664398579a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:54.233926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1246/2509] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:54.240420 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cf43b-129a-47d0-9004-def0b5d6c447 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.253368 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35cf43b-129a-47d0-9004-def0b5d6c447 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.261161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c2e78-cc6a-4b50-b9e7-6aac3ffb336f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.261868 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c2e78-cc6a-4b50-b9e7-6aac3ffb336f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.262158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c2e78-cc6a-4b50-b9e7-6aac3ffb336f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.262439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c2e78-cc6a-4b50-b9e7-6aac3ffb336f admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.263187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c2e78-cc6a-4b50-b9e7-6aac3ffb336f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:35:54.263499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c2e78-cc6a-4b50-b9e7-6aac3ffb336f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.265141 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3c2e78-cc6a-4b50-b9e7-6aac3ffb336f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.278808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1264/2510] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:35:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:35:54.286388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf607438-a59c-4e9c-849e-55664398579a None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.296373 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf607438-a59c-4e9c-849e-55664398579a None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.301694 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70e786-e42f-45b6-9d5a-ed109638f3ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.302407 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70e786-e42f-45b6-9d5a-ed109638f3ee admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.302773 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70e786-e42f-45b6-9d5a-ed109638f3ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.303132 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e70e786-e42f-45b6-9d5a-ed109638f3ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.303910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70e786-e42f-45b6-9d5a-ed109638f3ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.306021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e70e786-e42f-45b6-9d5a-ed109638f3ee admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.341283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1247/2511] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:35:54 2022] POST /identity/v3/projects => generated 367 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:54.343470 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c2e78-cc6a-4b50-b9e7-6aac3ffb336f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.354960 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c2e78-cc6a-4b50-b9e7-6aac3ffb336f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.383777 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:54.420808 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ec682-4d10-4581-8e85-9e4ed3309405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.421615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ec682-4d10-4581-8e85-9e4ed3309405 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.421903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ec682-4d10-4581-8e85-9e4ed3309405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.422187 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620ec682-4d10-4581-8e85-9e4ed3309405 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.423176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ec682-4d10-4581-8e85-9e4ed3309405 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.425843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620ec682-4d10-4581-8e85-9e4ed3309405 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.456640 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1265/2512] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:35:54 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:35:54.461731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70e786-e42f-45b6-9d5a-ed109638f3ee admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.475179 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e70e786-e42f-45b6-9d5a-ed109638f3ee admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.480049 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9683f-c343-450c-8614-360bb82e3849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.480734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9683f-c343-450c-8614-360bb82e3849 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.481024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9683f-c343-450c-8614-360bb82e3849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.481306 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9683f-c343-450c-8614-360bb82e3849 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.482283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9683f-c343-450c-8614-360bb82e3849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.483898 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb9683f-c343-450c-8614-360bb82e3849 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.499990 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1248/2513] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:54.504828 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ec682-4d10-4581-8e85-9e4ed3309405 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.516256 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620ec682-4d10-4581-8e85-9e4ed3309405 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.523087 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3412b86-b8e3-478b-b1ed-d4b33403b6e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.523768 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3412b86-b8e3-478b-b1ed-d4b33403b6e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.524054 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3412b86-b8e3-478b-b1ed-d4b33403b6e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.524332 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3412b86-b8e3-478b-b1ed-d4b33403b6e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.525103 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3412b86-b8e3-478b-b1ed-d4b33403b6e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.526754 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3412b86-b8e3-478b-b1ed-d4b33403b6e0 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.538313 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1266/2514] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:54.542690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9683f-c343-450c-8614-360bb82e3849 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.563105 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9683f-c343-450c-8614-360bb82e3849 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.575096 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92187948-afc9-4245-a5d7-56cbd91fc6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.576001 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92187948-afc9-4245-a5d7-56cbd91fc6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.576289 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92187948-afc9-4245-a5d7-56cbd91fc6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.576567 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92187948-afc9-4245-a5d7-56cbd91fc6a3 admin admin] PATH_INFO: `/v3/projects/9df7b0bc12a84104ba5dafe1be2306ba/users/30fc3a886c3d4711ad7f779f3f212d8e/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.598976 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92187948-afc9-4245-a5d7-56cbd91fc6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df7b0bc12a84104ba5dafe1be2306ba, user_id=30fc3a886c3d4711ad7f779f3f212d8e, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.601344 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92187948-afc9-4245-a5d7-56cbd91fc6a3 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.617509 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1249/2515] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:54 2022] PUT /identity/v3/projects/9df7b0bc12a84104ba5dafe1be2306ba/users/30fc3a886c3d4711ad7f779f3f212d8e/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:54.628603 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d03048-71b1-4ff6-a08e-3d288b270fb7 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.628898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d03048-71b1-4ff6-a08e-3d288b270fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.629191 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d03048-71b1-4ff6-a08e-3d288b270fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.658065 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-70d03048-71b1-4ff6-a08e-3d288b270fb7 None None] MFA Rules not processed for user `30fc3a886c3d4711ad7f779f3f212d8e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:54.660162 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-70d03048-71b1-4ff6-a08e-3d288b270fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:54.737435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1267/2516] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:54 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:54.742151 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92187948-afc9-4245-a5d7-56cbd91fc6a3 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.813753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92187948-afc9-4245-a5d7-56cbd91fc6a3 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.819027 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21696e5-2ad7-4505-91b8-d0c554ef01d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.819701 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21696e5-2ad7-4505-91b8-d0c554ef01d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.819989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21696e5-2ad7-4505-91b8-d0c554ef01d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.820267 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21696e5-2ad7-4505-91b8-d0c554ef01d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.820999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21696e5-2ad7-4505-91b8-d0c554ef01d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.822958 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21696e5-2ad7-4505-91b8-d0c554ef01d9 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.853328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1250/2517] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:35:54 2022] POST /identity/v3/projects => generated 367 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:54.858574 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d03048-71b1-4ff6-a08e-3d288b270fb7 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.869767 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d03048-71b1-4ff6-a08e-3d288b270fb7 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.928240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04689bb5-add9-4917-a4a2-cacf17d274de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.928921 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04689bb5-add9-4917-a4a2-cacf17d274de admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.929207 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04689bb5-add9-4917-a4a2-cacf17d274de admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.929516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04689bb5-add9-4917-a4a2-cacf17d274de admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.930231 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04689bb5-add9-4917-a4a2-cacf17d274de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.932159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04689bb5-add9-4917-a4a2-cacf17d274de admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.957030 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1268/2518] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:35:54 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:54.958331 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21696e5-2ad7-4505-91b8-d0c554ef01d9 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:54.969585 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21696e5-2ad7-4505-91b8-d0c554ef01d9 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:54.975607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812afbe-74df-48dd-9544-2c125a4b54a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:54.976293 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812afbe-74df-48dd-9544-2c125a4b54a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:54.976579 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812afbe-74df-48dd-9544-2c125a4b54a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:54.976857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9812afbe-74df-48dd-9544-2c125a4b54a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:54.977636 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812afbe-74df-48dd-9544-2c125a4b54a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:54.979264 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9812afbe-74df-48dd-9544-2c125a4b54a5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:54.990927 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1251/2519] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:54.995176 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04689bb5-add9-4917-a4a2-cacf17d274de admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:55.006947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04689bb5-add9-4917-a4a2-cacf17d274de admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:55.011665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94d80d-3752-435c-ae25-76186e685ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:55.012554 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94d80d-3752-435c-ae25-76186e685ee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:55.012840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94d80d-3752-435c-ae25-76186e685ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:55.013119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94d80d-3752-435c-ae25-76186e685ee2 admin admin] PATH_INFO: `/v3/projects/4fd1c1e038cd445c96bfa6bc38eeb808/users/ab6f3cb17b7b4c358928ecf4f4bc031c/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:55.030914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94d80d-3752-435c-ae25-76186e685ee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd1c1e038cd445c96bfa6bc38eeb808, user_id=ab6f3cb17b7b4c358928ecf4f4bc031c, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:55.034241 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94d80d-3752-435c-ae25-76186e685ee2 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:55.048810 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1269/2520] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:54 2022] PUT /identity/v3/projects/4fd1c1e038cd445c96bfa6bc38eeb808/users/ab6f3cb17b7b4c358928ecf4f4bc031c/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:55.053037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812afbe-74df-48dd-9544-2c125a4b54a5 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:55.063409 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9812afbe-74df-48dd-9544-2c125a4b54a5 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:55.088843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:55.133918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960eb76e-2cc8-4900-8655-5df5041588f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:55.134602 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eb76e-2cc8-4900-8655-5df5041588f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:55.134888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eb76e-2cc8-4900-8655-5df5041588f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:55.135168 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960eb76e-2cc8-4900-8655-5df5041588f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:55.135897 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960eb76e-2cc8-4900-8655-5df5041588f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:55.137560 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960eb76e-2cc8-4900-8655-5df5041588f4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:55.148530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1252/2521] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:55 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:55.152689 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94d80d-3752-435c-ae25-76186e685ee2 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:55.158240 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:55.167068 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94d80d-3752-435c-ae25-76186e685ee2 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:55.172102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140f593-fbe9-416a-9301-b016825af334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:55.172998 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140f593-fbe9-416a-9301-b016825af334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:55.173284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140f593-fbe9-416a-9301-b016825af334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:55.173588 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140f593-fbe9-416a-9301-b016825af334 admin admin] PATH_INFO: `/v3/projects/4fd1c1e038cd445c96bfa6bc38eeb808/users/ab6f3cb17b7b4c358928ecf4f4bc031c/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:55.175591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140f593-fbe9-416a-9301-b016825af334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd1c1e038cd445c96bfa6bc38eeb808, user_id=ab6f3cb17b7b4c358928ecf4f4bc031c, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:55.178294 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140f593-fbe9-416a-9301-b016825af334 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:55.189432 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1270/2522] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:55 2022] PUT /identity/v3/projects/4fd1c1e038cd445c96bfa6bc38eeb808/users/ab6f3cb17b7b4c358928ecf4f4bc031c/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:55.200477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7799fd-df6f-4613-bfca-5b4ab0b0ad01 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:55.200770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7799fd-df6f-4613-bfca-5b4ab0b0ad01 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:55.201076 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7799fd-df6f-4613-bfca-5b4ab0b0ad01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:55.231972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-5a7799fd-df6f-4613-bfca-5b4ab0b0ad01 None None] MFA Rules not processed for user `ab6f3cb17b7b4c358928ecf4f4bc031c`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:55.233200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-5a7799fd-df6f-4613-bfca-5b4ab0b0ad01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:55.311181 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1253/2523] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:55 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:55.324153 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140f593-fbe9-416a-9301-b016825af334 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:55.325938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c140f593-fbe9-416a-9301-b016825af334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:55.388969 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140f593-fbe9-416a-9301-b016825af334 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:55.392761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8ae45f-d6ba-48a3-97c0-811f891e0394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:55.394338 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ae45f-d6ba-48a3-97c0-811f891e0394 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:55.394627 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ae45f-d6ba-48a3-97c0-811f891e0394 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:55.394903 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8ae45f-d6ba-48a3-97c0-811f891e0394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:55.404744 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ae45f-d6ba-48a3-97c0-811f891e0394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:55.406230 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8ae45f-d6ba-48a3-97c0-811f891e0394 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:55.416747 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1271/2524] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:55 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:56.858803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7799fd-df6f-4613-bfca-5b4ab0b0ad01 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:56.935766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7799fd-df6f-4613-bfca-5b4ab0b0ad01 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:56.940801 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:56.941720 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:56.942277 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:56.942558 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] PATH_INFO: `/v3/users/30fc3a886c3d4711ad7f779f3f212d8e` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:56.943862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30fc3a886c3d4711ad7f779f3f212d8e)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:56.945814 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:56.998674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30fc3a886c3d4711ad7f779f3f212d8e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.000166 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:57.008137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30fc3a886c3d4711ad7f779f3f212d8e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.009133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30fc3a886c3d4711ad7f779f3f212d8e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.030341 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30fc3a886c3d4711ad7f779f3f212d8e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.042787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30fc3a886c3d4711ad7f779f3f212d8e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.052211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1254/2525] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:35:56 2022] DELETE /identity/v3/users/30fc3a886c3d4711ad7f779f3f212d8e => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:57.058971 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca573eb0-0ec4-41a5-bb9f-662bb8976a8c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:57.059533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca573eb0-0ec4-41a5-bb9f-662bb8976a8c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:57.059631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca573eb0-0ec4-41a5-bb9f-662bb8976a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:57.083403 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ca573eb0-0ec4-41a5-bb9f-662bb8976a8c None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:57.085502 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ca573eb0-0ec4-41a5-bb9f-662bb8976a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:57.151478 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1272/2526] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:57.161324 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:57.162906 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:57.222440 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22281639-2511-4806-9271-eb387f87fda1 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:57.226325 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617d0b9-b40e-4ad5-b8a8-75d0d6f71289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:57.226981 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617d0b9-b40e-4ad5-b8a8-75d0d6f71289 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:57.227262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617d0b9-b40e-4ad5-b8a8-75d0d6f71289 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:57.227532 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7617d0b9-b40e-4ad5-b8a8-75d0d6f71289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:57.238257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617d0b9-b40e-4ad5-b8a8-75d0d6f71289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:57.239705 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7617d0b9-b40e-4ad5-b8a8-75d0d6f71289 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:57.250339 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1255/2527] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:57.538644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca573eb0-0ec4-41a5-bb9f-662bb8976a8c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:57.540108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ca573eb0-0ec4-41a5-bb9f-662bb8976a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:57.555465 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca573eb0-0ec4-41a5-bb9f-662bb8976a8c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:57.560161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:57.561120 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:57.561435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:57.561717 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] PATH_INFO: `/v3/projects/9df7b0bc12a84104ba5dafe1be2306ba` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:57.562983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df7b0bc12a84104ba5dafe1be2306ba)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:57.565620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:57.621914 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df7b0bc12a84104ba5dafe1be2306ba', 'request_id': 'req-2740344e-e217-4544-aa78-33890c56ad3b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.629645 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df7b0bc12a84104ba5dafe1be2306ba', 'request_id': 'req-2740344e-e217-4544-aa78-33890c56ad3b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.638538 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df7b0bc12a84104ba5dafe1be2306ba', 'request_id': 'req-2740344e-e217-4544-aa78-33890c56ad3b'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.650721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] The token cache is being invalidate because project 9df7b0bc12a84104ba5dafe1be2306ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:35:57.651013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.651679 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.653304 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1273/2528] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:35:57 2022] DELETE /identity/v3/projects/9df7b0bc12a84104ba5dafe1be2306ba => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:57.657361 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617d0b9-b40e-4ad5-b8a8-75d0d6f71289 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:57.668248 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7617d0b9-b40e-4ad5-b8a8-75d0d6f71289 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:57.729427 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:57.730221 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:57.730510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:57.730787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] PATH_INFO: `/v3/users/ab6f3cb17b7b4c358928ecf4f4bc031c` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:57.732084 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab6f3cb17b7b4c358928ecf4f4bc031c)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:57.734089 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:57.782771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab6f3cb17b7b4c358928ecf4f4bc031c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.791141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab6f3cb17b7b4c358928ecf4f4bc031c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.791896 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab6f3cb17b7b4c358928ecf4f4bc031c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.803980 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:57.806622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab6f3cb17b7b4c358928ecf4f4bc031c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.816227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-3f4d5fab-e2d0-4f20-ab6b-36fa80751ead admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab6f3cb17b7b4c358928ecf4f4bc031c'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:57.825783 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1256/2529] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:35:57 2022] DELETE /identity/v3/users/ab6f3cb17b7b4c358928ecf4f4bc031c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:58.117228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:58.121938 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:58.137729 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2740344e-e217-4544-aa78-33890c56ad3b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:58.152811 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:58.214349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:58.215234 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.215520 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.215798 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] PATH_INFO: `/v3/projects/4fd1c1e038cd445c96bfa6bc38eeb808` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.217079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd1c1e038cd445c96bfa6bc38eeb808)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:58.219065 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:58.286108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd1c1e038cd445c96bfa6bc38eeb808', 'request_id': 'req-05f1e499-2d87-4dc6-adf6-864d8a40a119'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:58.293344 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd1c1e038cd445c96bfa6bc38eeb808', 'request_id': 'req-05f1e499-2d87-4dc6-adf6-864d8a40a119'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:58.302582 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd1c1e038cd445c96bfa6bc38eeb808', 'request_id': 'req-05f1e499-2d87-4dc6-adf6-864d8a40a119'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:58.315172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] The token cache is being invalidate because project 4fd1c1e038cd445c96bfa6bc38eeb808 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80288) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:35:58.315578 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:58.316273 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-05f1e499-2d87-4dc6-adf6-864d8a40a119 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:58.318158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1274/2530] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:35:58 2022] DELETE /identity/v3/projects/4fd1c1e038cd445c96bfa6bc38eeb808 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:58.323892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbeedcb-494c-4b64-bcab-3f3e0f181500 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.324450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbeedcb-494c-4b64-bcab-3f3e0f181500 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.324881 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbeedcb-494c-4b64-bcab-3f3e0f181500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.353575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-cbbeedcb-494c-4b64-bcab-3f3e0f181500 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:58.355322 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-cbbeedcb-494c-4b64-bcab-3f3e0f181500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:58.459888 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1257/2531] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:35:58.473591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6f236-3fbc-4a57-b176-7e501a731ac8 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.473888 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6f236-3fbc-4a57-b176-7e501a731ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.474179 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6f236-3fbc-4a57-b176-7e501a731ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.494024 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-78b6f236-3fbc-4a57-b176-7e501a731ac8 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:58.495173 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-78b6f236-3fbc-4a57-b176-7e501a731ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:58.504372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1275/2532] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:58.508803 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbeedcb-494c-4b64-bcab-3f3e0f181500 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:58.519457 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbeedcb-494c-4b64-bcab-3f3e0f181500 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:58.524203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e71a9-11db-4106-833b-071f2ea788ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:58.524935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e71a9-11db-4106-833b-071f2ea788ca admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.525217 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e71a9-11db-4106-833b-071f2ea788ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.525525 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e71a9-11db-4106-833b-071f2ea788ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.526260 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e71a9-11db-4106-833b-071f2ea788ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80287) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:35:58.526568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e71a9-11db-4106-833b-071f2ea788ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:58.528161 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e71a9-11db-4106-833b-071f2ea788ca admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:58.541648 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1258/2533] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:35:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 12 13:35:58.546163 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6f236-3fbc-4a57-b176-7e501a731ac8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:58.556745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6f236-3fbc-4a57-b176-7e501a731ac8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:58.561743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbdcbf-fcc2-4b96-a3df-7e9bbb3ad962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:58.562407 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbdcbf-fcc2-4b96-a3df-7e9bbb3ad962 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.562691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbdcbf-fcc2-4b96-a3df-7e9bbb3ad962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.562963 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dbdcbf-fcc2-4b96-a3df-7e9bbb3ad962 admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.563652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbdcbf-fcc2-4b96-a3df-7e9bbb3ad962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:58.566168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dbdcbf-fcc2-4b96-a3df-7e9bbb3ad962 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:58.600511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1276/2534] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:35:58 2022] POST /identity/v3/projects => generated 387 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:58.605202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e71a9-11db-4106-833b-071f2ea788ca admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:58.617390 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e71a9-11db-4106-833b-071f2ea788ca admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:58.681749 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4732691-1198-4257-b764-4df346540dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:58.682460 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4732691-1198-4257-b764-4df346540dcd admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.682737 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4732691-1198-4257-b764-4df346540dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.683019 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4732691-1198-4257-b764-4df346540dcd admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.683732 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4732691-1198-4257-b764-4df346540dcd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:58.685719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4732691-1198-4257-b764-4df346540dcd admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:58.708685 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1259/2535] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:35:58 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:58.712742 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbdcbf-fcc2-4b96-a3df-7e9bbb3ad962 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:58.724532 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dbdcbf-fcc2-4b96-a3df-7e9bbb3ad962 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:58.729709 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e050eab-28dd-4106-b2f0-a96eb787dd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:58.730391 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e050eab-28dd-4106-b2f0-a96eb787dd4a admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.730676 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e050eab-28dd-4106-b2f0-a96eb787dd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.730955 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e050eab-28dd-4106-b2f0-a96eb787dd4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.731694 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e050eab-28dd-4106-b2f0-a96eb787dd4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:58.737785 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e050eab-28dd-4106-b2f0-a96eb787dd4a admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:58.749612 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1277/2536] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:58.754263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4732691-1198-4257-b764-4df346540dcd admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:58.766474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4732691-1198-4257-b764-4df346540dcd admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:58.771416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70c48a-33d1-4b06-955a-76037e864e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:58.772101 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70c48a-33d1-4b06-955a-76037e864e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.772394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70c48a-33d1-4b06-955a-76037e864e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.772679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70c48a-33d1-4b06-955a-76037e864e9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.773479 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70c48a-33d1-4b06-955a-76037e864e9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:58.775104 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70c48a-33d1-4b06-955a-76037e864e9f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:58.786938 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1260/2537] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:58.791215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e050eab-28dd-4106-b2f0-a96eb787dd4a admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:58.804999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e050eab-28dd-4106-b2f0-a96eb787dd4a admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:58.809696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c01e3-db7d-42a9-bb56-a0f930648600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:58.810596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c01e3-db7d-42a9-bb56-a0f930648600 admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.810909 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c01e3-db7d-42a9-bb56-a0f930648600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.811211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c01e3-db7d-42a9-bb56-a0f930648600 admin admin] PATH_INFO: `/v3/projects/c92ad045a4b249c7abcce37774ce0ab2/users/2eb0ec8a6c8449dab0f35b4984089380/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.827354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c01e3-db7d-42a9-bb56-a0f930648600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c92ad045a4b249c7abcce37774ce0ab2, user_id=2eb0ec8a6c8449dab0f35b4984089380, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:58.830119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c01e3-db7d-42a9-bb56-a0f930648600 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:58.846516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1278/2538] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:58 2022] PUT /identity/v3/projects/c92ad045a4b249c7abcce37774ce0ab2/users/2eb0ec8a6c8449dab0f35b4984089380/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:58.858024 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74d1cb-7a32-489a-b841-1e91e8a84015 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:58.858325 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74d1cb-7a32-489a-b841-1e91e8a84015 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:58.858610 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74d1cb-7a32-489a-b841-1e91e8a84015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:58.889836 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-fc74d1cb-7a32-489a-b841-1e91e8a84015 None None] MFA Rules not processed for user `2eb0ec8a6c8449dab0f35b4984089380`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:58.891074 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-fc74d1cb-7a32-489a-b841-1e91e8a84015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:58.934997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:58.961451 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1261/2539] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:58.966138 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c01e3-db7d-42a9-bb56-a0f930648600 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.018058 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:59.039665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c01e3-db7d-42a9-bb56-a0f930648600 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:59.044439 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8184d28-b3f7-49f2-b127-87b630f3736e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:59.045131 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8184d28-b3f7-49f2-b127-87b630f3736e admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.045442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8184d28-b3f7-49f2-b127-87b630f3736e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.045721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8184d28-b3f7-49f2-b127-87b630f3736e admin admin] PATH_INFO: `/v3/projects` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.046417 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8184d28-b3f7-49f2-b127-87b630f3736e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:59.048319 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8184d28-b3f7-49f2-b127-87b630f3736e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:59.077723 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1279/2540] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:35:58 2022] POST /identity/v3/projects => generated 385 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:35:59.082346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74d1cb-7a32-489a-b841-1e91e8a84015 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.095772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74d1cb-7a32-489a-b841-1e91e8a84015 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:59.157173 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ffab4-547d-4c05-a453-fcd718bd703f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:59.157890 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ffab4-547d-4c05-a453-fcd718bd703f admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.158258 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ffab4-547d-4c05-a453-fcd718bd703f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.158444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ffab4-547d-4c05-a453-fcd718bd703f admin admin] PATH_INFO: `/v3/users` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.159468 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ffab4-547d-4c05-a453-fcd718bd703f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:59.161693 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ffab4-547d-4c05-a453-fcd718bd703f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:59.190912 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1262/2541] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:35:59 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:35:59.195250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8184d28-b3f7-49f2-b127-87b630f3736e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.205846 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8184d28-b3f7-49f2-b127-87b630f3736e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:59.210695 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cab759-a91d-4d9f-8bd9-df7f79e80f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:59.211358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cab759-a91d-4d9f-8bd9-df7f79e80f6b admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.211641 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cab759-a91d-4d9f-8bd9-df7f79e80f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.211913 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cab759-a91d-4d9f-8bd9-df7f79e80f6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.212677 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cab759-a91d-4d9f-8bd9-df7f79e80f6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:59.214481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cab759-a91d-4d9f-8bd9-df7f79e80f6b admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:59.226001 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1280/2542] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:35:59.230216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ffab4-547d-4c05-a453-fcd718bd703f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.241661 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ffab4-547d-4c05-a453-fcd718bd703f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:59.246499 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ae9a-9e91-411c-bd27-15fd14a18946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:59.247408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ae9a-9e91-411c-bd27-15fd14a18946 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.247702 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ae9a-9e91-411c-bd27-15fd14a18946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.247978 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f4ae9a-9e91-411c-bd27-15fd14a18946 admin admin] PATH_INFO: `/v3/projects/9909c88bebba496d85fb7cac3c9a254a/users/ff86f9db0190421ba9598bea9f333b75/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.265753 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4ae9a-9e91-411c-bd27-15fd14a18946 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9909c88bebba496d85fb7cac3c9a254a, user_id=ff86f9db0190421ba9598bea9f333b75, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:59.268872 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f4ae9a-9e91-411c-bd27-15fd14a18946 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:59.282646 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1263/2543] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:59 2022] PUT /identity/v3/projects/9909c88bebba496d85fb7cac3c9a254a/users/ff86f9db0190421ba9598bea9f333b75/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:35:59.287366 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cab759-a91d-4d9f-8bd9-df7f79e80f6b admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.297633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cab759-a91d-4d9f-8bd9-df7f79e80f6b admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:59.369967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924e259-b8cf-40b6-8808-78f4ec536551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:59.371192 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924e259-b8cf-40b6-8808-78f4ec536551 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.371703 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924e259-b8cf-40b6-8808-78f4ec536551 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.372237 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924e259-b8cf-40b6-8808-78f4ec536551 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.373579 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924e259-b8cf-40b6-8808-78f4ec536551 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:59.375596 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1924e259-b8cf-40b6-8808-78f4ec536551 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:59.393105 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1281/2544] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:35:59 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 12 13:35:59.397852 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ae9a-9e91-411c-bd27-15fd14a18946 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.410085 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f4ae9a-9e91-411c-bd27-15fd14a18946 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:59.414999 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6326236-af4f-4fce-9b79-6fcee6f06010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:59.416302 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6326236-af4f-4fce-9b79-6fcee6f06010 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.416432 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6326236-af4f-4fce-9b79-6fcee6f06010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.416623 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6326236-af4f-4fce-9b79-6fcee6f06010 admin admin] PATH_INFO: `/v3/projects/9909c88bebba496d85fb7cac3c9a254a/users/ff86f9db0190421ba9598bea9f333b75/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.418639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6326236-af4f-4fce-9b79-6fcee6f06010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9909c88bebba496d85fb7cac3c9a254a, user_id=ff86f9db0190421ba9598bea9f333b75, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:59.421770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6326236-af4f-4fce-9b79-6fcee6f06010 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:59.433314 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1264/2545] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:35:59 2022] PUT /identity/v3/projects/9909c88bebba496d85fb7cac3c9a254a/users/ff86f9db0190421ba9598bea9f333b75/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:59.446250 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d987bd-8d50-4a15-bc36-38482a799b3c None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.446540 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d987bd-8d50-4a15-bc36-38482a799b3c None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.446818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d987bd-8d50-4a15-bc36-38482a799b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.475925 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-74d987bd-8d50-4a15-bc36-38482a799b3c None None] MFA Rules not processed for user `ff86f9db0190421ba9598bea9f333b75`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:59.477158 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-74d987bd-8d50-4a15-bc36-38482a799b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:59.556114 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1282/2546] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:59.568316 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6326236-af4f-4fce-9b79-6fcee6f06010 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.570053 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-e6326236-af4f-4fce-9b79-6fcee6f06010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:59.629790 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6326236-af4f-4fce-9b79-6fcee6f06010 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:59.633539 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41536a2-32dc-4af2-bcfc-f415eeb4ce8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:59.634200 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41536a2-32dc-4af2-bcfc-f415eeb4ce8a service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.634490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41536a2-32dc-4af2-bcfc-f415eeb4ce8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.634767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41536a2-32dc-4af2-bcfc-f415eeb4ce8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.645323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41536a2-32dc-4af2-bcfc-f415eeb4ce8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:59.647083 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41536a2-32dc-4af2-bcfc-f415eeb4ce8a service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:59.657183 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1265/2547] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:59 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:35:59.687544 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d987bd-8d50-4a15-bc36-38482a799b3c None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.731911 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:59.754799 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d987bd-8d50-4a15-bc36-38482a799b3c None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:35:59.760358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:35:59.761521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.762051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.762546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] PATH_INFO: `/v3/users/2eb0ec8a6c8449dab0f35b4984089380` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.764330 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2eb0ec8a6c8449dab0f35b4984089380)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:35:59.766340 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:35:59.817737 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2eb0ec8a6c8449dab0f35b4984089380'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:59.824849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2eb0ec8a6c8449dab0f35b4984089380'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:59.825581 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2eb0ec8a6c8449dab0f35b4984089380'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:59.834048 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2eb0ec8a6c8449dab0f35b4984089380'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:59.846229 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2eb0ec8a6c8449dab0f35b4984089380'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:35:59.852315 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1283/2548] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:35:59 2022] DELETE /identity/v3/users/2eb0ec8a6c8449dab0f35b4984089380 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:35:59.857092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e488c06-8d95-4dc3-b85c-b85e9a5d4d63 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:35:59.857453 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e488c06-8d95-4dc3-b85c-b85e9a5d4d63 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:35:59.857767 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e488c06-8d95-4dc3-b85c-b85e9a5d4d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:35:59.876925 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-6e488c06-8d95-4dc3-b85c-b85e9a5d4d63 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:35:59.879607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6e488c06-8d95-4dc3-b85c-b85e9a5d4d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:35:59.917731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:35:59.953300 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1266/2549] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:35:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:35:59.963227 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:35:59.964750 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:00.023621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5155110c-8601-4acc-959d-a6dc4f81ada3 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:00.028890 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ca294-2c17-44b0-90ef-bddfb9ed691e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:00.029967 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca294-2c17-44b0-90ef-bddfb9ed691e service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:00.030614 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca294-2c17-44b0-90ef-bddfb9ed691e service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:00.031075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9ca294-2c17-44b0-90ef-bddfb9ed691e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:00.050927 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ca294-2c17-44b0-90ef-bddfb9ed691e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:00.052435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9ca294-2c17-44b0-90ef-bddfb9ed691e service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:00.063260 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1284/2550] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:35:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:00.358069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e488c06-8d95-4dc3-b85c-b85e9a5d4d63 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:00.359517 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-6e488c06-8d95-4dc3-b85c-b85e9a5d4d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:00.371974 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e488c06-8d95-4dc3-b85c-b85e9a5d4d63 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:00.376988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:00.378497 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:00.378772 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:00.379057 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] PATH_INFO: `/v3/projects/c92ad045a4b249c7abcce37774ce0ab2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:00.380337 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c92ad045a4b249c7abcce37774ce0ab2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:00.382357 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:00.431152 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c92ad045a4b249c7abcce37774ce0ab2', 'request_id': 'req-51c97d07-0eec-457b-9260-7c3a2e255044'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.437271 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c92ad045a4b249c7abcce37774ce0ab2', 'request_id': 'req-51c97d07-0eec-457b-9260-7c3a2e255044'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.449013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c92ad045a4b249c7abcce37774ce0ab2', 'request_id': 'req-51c97d07-0eec-457b-9260-7c3a2e255044'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.462387 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] The token cache is being invalidate because project c92ad045a4b249c7abcce37774ce0ab2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:36:00.462655 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.463365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.464924 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1267/2551] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:36:00 2022] DELETE /identity/v3/projects/c92ad045a4b249c7abcce37774ce0ab2 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:36:00.468962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ca294-2c17-44b0-90ef-bddfb9ed691e service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:00.480243 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9ca294-2c17-44b0-90ef-bddfb9ed691e service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:00.542821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:00.543692 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:00.543979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:00.544261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] PATH_INFO: `/v3/users/ff86f9db0190421ba9598bea9f333b75` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:00.545631 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff86f9db0190421ba9598bea9f333b75)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:00.547479 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:00.550628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:00.614311 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff86f9db0190421ba9598bea9f333b75'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.621909 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff86f9db0190421ba9598bea9f333b75'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.622687 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff86f9db0190421ba9598bea9f333b75'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.632067 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff86f9db0190421ba9598bea9f333b75'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.646351 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-ed434ec4-d2f6-4df3-b58e-bc01ca9e4498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff86f9db0190421ba9598bea9f333b75'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:00.652905 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1285/2552] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:36:00 2022] DELETE /identity/v3/users/ff86f9db0190421ba9598bea9f333b75 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:36:00.944204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:00.945764 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:00.959787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c97d07-0eec-457b-9260-7c3a2e255044 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.025068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.026080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.026362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.026635 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] PATH_INFO: `/v3/projects/9909c88bebba496d85fb7cac3c9a254a` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.027880 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9909c88bebba496d85fb7cac3c9a254a)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.029834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.078467 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9909c88bebba496d85fb7cac3c9a254a', 'request_id': 'req-b0d69439-89b1-44ff-bfde-112b6171dc43'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:01.084048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9909c88bebba496d85fb7cac3c9a254a', 'request_id': 'req-b0d69439-89b1-44ff-bfde-112b6171dc43'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:01.088048 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:01.092834 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9909c88bebba496d85fb7cac3c9a254a', 'request_id': 'req-b0d69439-89b1-44ff-bfde-112b6171dc43'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:01.106771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] The token cache is being invalidate because project 9909c88bebba496d85fb7cac3c9a254a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:36:01.107062 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:01.107710 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-b0d69439-89b1-44ff-bfde-112b6171dc43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:01.109239 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1268/2553] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:36:00 2022] DELETE /identity/v3/projects/9909c88bebba496d85fb7cac3c9a254a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:36:01.119007 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97967e47-9a60-4701-803c-4b3c700fc279 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.119306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97967e47-9a60-4701-803c-4b3c700fc279 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.119594 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97967e47-9a60-4701-803c-4b3c700fc279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.138562 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-97967e47-9a60-4701-803c-4b3c700fc279 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:01.139705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-97967e47-9a60-4701-803c-4b3c700fc279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:01.202477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1286/2554] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:01.210547 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a961f4-e447-476f-98c1-e82a43c66d44 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.210865 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a961f4-e447-476f-98c1-e82a43c66d44 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.211172 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a961f4-e447-476f-98c1-e82a43c66d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.233237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b2a961f4-e447-476f-98c1-e82a43c66d44 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:01.234510 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b2a961f4-e447-476f-98c1-e82a43c66d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:01.243843 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1269/2555] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:01.248508 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97967e47-9a60-4701-803c-4b3c700fc279 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.259853 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97967e47-9a60-4701-803c-4b3c700fc279 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.266370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96792cf0-ff41-41bb-8f70-9502a9f97270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.267053 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96792cf0-ff41-41bb-8f70-9502a9f97270 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.267343 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96792cf0-ff41-41bb-8f70-9502a9f97270 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.267620 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96792cf0-ff41-41bb-8f70-9502a9f97270 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.269130 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96792cf0-ff41-41bb-8f70-9502a9f97270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:36:01.269485 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96792cf0-ff41-41bb-8f70-9502a9f97270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.271133 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96792cf0-ff41-41bb-8f70-9502a9f97270 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.283934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1287/2556] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:36:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:36:01.288346 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a961f4-e447-476f-98c1-e82a43c66d44 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.298007 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a961f4-e447-476f-98c1-e82a43c66d44 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.304243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c60f3-1c20-49a4-bc9b-0bd8601f9e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.304914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c60f3-1c20-49a4-bc9b-0bd8601f9e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.305202 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c60f3-1c20-49a4-bc9b-0bd8601f9e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.305512 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3c60f3-1c20-49a4-bc9b-0bd8601f9e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.306224 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c60f3-1c20-49a4-bc9b-0bd8601f9e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.308138 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3c60f3-1c20-49a4-bc9b-0bd8601f9e98 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.336920 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1270/2557] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:36:01 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:36:01.341049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96792cf0-ff41-41bb-8f70-9502a9f97270 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.351843 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96792cf0-ff41-41bb-8f70-9502a9f97270 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.413741 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db45755-273d-488e-97c8-709eed6f6695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.414367 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db45755-273d-488e-97c8-709eed6f6695 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.414650 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db45755-273d-488e-97c8-709eed6f6695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.414931 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db45755-273d-488e-97c8-709eed6f6695 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.415649 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db45755-273d-488e-97c8-709eed6f6695 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.417693 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db45755-273d-488e-97c8-709eed6f6695 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.428617 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:01.446046 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1288/2558] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:36:01 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:36:01.449323 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c60f3-1c20-49a4-bc9b-0bd8601f9e98 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.460393 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3c60f3-1c20-49a4-bc9b-0bd8601f9e98 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.465823 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82295b-eedb-4215-845f-2cbdb1e65bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.466495 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82295b-eedb-4215-845f-2cbdb1e65bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.466809 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82295b-eedb-4215-845f-2cbdb1e65bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.467082 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c82295b-eedb-4215-845f-2cbdb1e65bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.467807 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82295b-eedb-4215-845f-2cbdb1e65bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.469433 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c82295b-eedb-4215-845f-2cbdb1e65bf6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.481284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1271/2559] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:36:01.485236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db45755-273d-488e-97c8-709eed6f6695 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.497075 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db45755-273d-488e-97c8-709eed6f6695 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.501793 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e351bf2-8708-40ff-97b0-73d1d70238af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.503168 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e351bf2-8708-40ff-97b0-73d1d70238af admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.503547 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e351bf2-8708-40ff-97b0-73d1d70238af admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.503826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e351bf2-8708-40ff-97b0-73d1d70238af admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.504569 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e351bf2-8708-40ff-97b0-73d1d70238af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.506215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e351bf2-8708-40ff-97b0-73d1d70238af admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.518599 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1289/2560] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:36:01.522984 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82295b-eedb-4215-845f-2cbdb1e65bf6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.537038 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c82295b-eedb-4215-845f-2cbdb1e65bf6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.542916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc9e75-8ba8-4929-aa16-7238c90c1697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.543931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc9e75-8ba8-4929-aa16-7238c90c1697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.544281 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc9e75-8ba8-4929-aa16-7238c90c1697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.544624 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc9e75-8ba8-4929-aa16-7238c90c1697 admin admin] PATH_INFO: `/v3/projects/3ef830774b5945be9c9618d1d1c683c2/users/3685a176c372466b92bbaa9cbacdea8d/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.560862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc9e75-8ba8-4929-aa16-7238c90c1697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ef830774b5945be9c9618d1d1c683c2, user_id=3685a176c372466b92bbaa9cbacdea8d, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.563602 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc9e75-8ba8-4929-aa16-7238c90c1697 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.577064 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1272/2561] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:36:01 2022] PUT /identity/v3/projects/3ef830774b5945be9c9618d1d1c683c2/users/3685a176c372466b92bbaa9cbacdea8d/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:36:01.589049 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bffe0-e59d-42ad-a6b0-c3791c4f428d None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.589346 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bffe0-e59d-42ad-a6b0-c3791c4f428d None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.589673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bffe0-e59d-42ad-a6b0-c3791c4f428d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.618761 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-bf7bffe0-e59d-42ad-a6b0-c3791c4f428d None None] MFA Rules not processed for user `3685a176c372466b92bbaa9cbacdea8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:01.619999 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-bf7bffe0-e59d-42ad-a6b0-c3791c4f428d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:01.685521 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1290/2562] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:01 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:01.690068 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc9e75-8ba8-4929-aa16-7238c90c1697 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.757639 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc9e75-8ba8-4929-aa16-7238c90c1697 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.762348 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a391f72-d4d7-485a-b0da-ded8a4ea41a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.763021 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a391f72-d4d7-485a-b0da-ded8a4ea41a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.763308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a391f72-d4d7-485a-b0da-ded8a4ea41a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.763590 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a391f72-d4d7-485a-b0da-ded8a4ea41a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.764285 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a391f72-d4d7-485a-b0da-ded8a4ea41a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.770928 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a391f72-d4d7-485a-b0da-ded8a4ea41a0 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.796529 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1273/2563] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:36:01 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:36:01.801813 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bffe0-e59d-42ad-a6b0-c3791c4f428d None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.812873 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bffe0-e59d-42ad-a6b0-c3791c4f428d None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.875816 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c20e7-e4a0-4f99-8768-d1effbfdc873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.876499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c20e7-e4a0-4f99-8768-d1effbfdc873 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.876782 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c20e7-e4a0-4f99-8768-d1effbfdc873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.877055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c20e7-e4a0-4f99-8768-d1effbfdc873 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.878659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c20e7-e4a0-4f99-8768-d1effbfdc873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.880511 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c20e7-e4a0-4f99-8768-d1effbfdc873 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.901891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1291/2564] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:36:01 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:36:01.905709 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a391f72-d4d7-485a-b0da-ded8a4ea41a0 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.916677 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a391f72-d4d7-485a-b0da-ded8a4ea41a0 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.921333 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727a1e4-f361-44a4-b818-0e6cd47128bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.922031 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727a1e4-f361-44a4-b818-0e6cd47128bc admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.922320 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727a1e4-f361-44a4-b818-0e6cd47128bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.922600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4727a1e4-f361-44a4-b818-0e6cd47128bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.923327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727a1e4-f361-44a4-b818-0e6cd47128bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.925750 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4727a1e4-f361-44a4-b818-0e6cd47128bc admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.936770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1274/2565] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:36:01.941064 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c20e7-e4a0-4f99-8768-d1effbfdc873 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:01.951434 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c20e7-e4a0-4f99-8768-d1effbfdc873 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:01.956823 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184e356-2a99-4eb2-b9ca-1a5a65ae51cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:01.957826 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184e356-2a99-4eb2-b9ca-1a5a65ae51cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:01.958108 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184e356-2a99-4eb2-b9ca-1a5a65ae51cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:01.958378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184e356-2a99-4eb2-b9ca-1a5a65ae51cc admin admin] PATH_INFO: `/v3/projects/9bf9f5c988944de5a20e72dcff854dfa/users/c5d2de7fedf948069cec95ee79752bba/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:01.974696 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184e356-2a99-4eb2-b9ca-1a5a65ae51cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf9f5c988944de5a20e72dcff854dfa, user_id=c5d2de7fedf948069cec95ee79752bba, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:01.977370 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184e356-2a99-4eb2-b9ca-1a5a65ae51cc admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:01.992161 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1292/2566] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:36:01 2022] PUT /identity/v3/projects/9bf9f5c988944de5a20e72dcff854dfa/users/c5d2de7fedf948069cec95ee79752bba/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:36:01.996187 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727a1e4-f361-44a4-b818-0e6cd47128bc admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:02.006793 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4727a1e4-f361-44a4-b818-0e6cd47128bc admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:02.070222 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7367ccb7-de55-469e-aef8-74e478e85361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:02.070904 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367ccb7-de55-469e-aef8-74e478e85361 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:02.071188 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367ccb7-de55-469e-aef8-74e478e85361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:02.071463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7367ccb7-de55-469e-aef8-74e478e85361 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:02.072193 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367ccb7-de55-469e-aef8-74e478e85361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:02.073825 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7367ccb7-de55-469e-aef8-74e478e85361 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:02.076092 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:02.086164 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1275/2567] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:36:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:36:02.090460 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184e356-2a99-4eb2-b9ca-1a5a65ae51cc admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:02.102014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184e356-2a99-4eb2-b9ca-1a5a65ae51cc admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:02.106724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf33ec-fd0a-47c1-ada2-52e3db3fffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:02.107619 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf33ec-fd0a-47c1-ada2-52e3db3fffcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:02.107904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf33ec-fd0a-47c1-ada2-52e3db3fffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:02.108182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaf33ec-fd0a-47c1-ada2-52e3db3fffcb admin admin] PATH_INFO: `/v3/projects/9bf9f5c988944de5a20e72dcff854dfa/users/c5d2de7fedf948069cec95ee79752bba/roles/8ef47d10f2ff4c33b9412cbb766cea3b` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:02.110212 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf33ec-fd0a-47c1-ada2-52e3db3fffcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf9f5c988944de5a20e72dcff854dfa, user_id=c5d2de7fedf948069cec95ee79752bba, role_id=8ef47d10f2ff4c33b9412cbb766cea3b)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:02.114222 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaf33ec-fd0a-47c1-ada2-52e3db3fffcb admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:02.125102 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1293/2568] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:36:02 2022] PUT /identity/v3/projects/9bf9f5c988944de5a20e72dcff854dfa/users/c5d2de7fedf948069cec95ee79752bba/roles/8ef47d10f2ff4c33b9412cbb766cea3b => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:36:02.193806 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a688453-6772-4bb2-b854-da98878fa1bf None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:02.194017 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a688453-6772-4bb2-b854-da98878fa1bf None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:02.194270 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a688453-6772-4bb2-b854-da98878fa1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:02.221322 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7a688453-6772-4bb2-b854-da98878fa1bf None None] MFA Rules not processed for user `c5d2de7fedf948069cec95ee79752bba`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:02.222603 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7a688453-6772-4bb2-b854-da98878fa1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:02.294383 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1276/2569] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:02.306035 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf33ec-fd0a-47c1-ada2-52e3db3fffcb admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:02.307979 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-baaf33ec-fd0a-47c1-ada2-52e3db3fffcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:02.342958 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:02.372618 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaf33ec-fd0a-47c1-ada2-52e3db3fffcb admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:02.376306 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41114d33-ba30-417e-870e-8d321dcf59a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:02.376965 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41114d33-ba30-417e-870e-8d321dcf59a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:02.377825 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41114d33-ba30-417e-870e-8d321dcf59a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:02.378126 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41114d33-ba30-417e-870e-8d321dcf59a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:02.388331 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41114d33-ba30-417e-870e-8d321dcf59a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:02.389805 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41114d33-ba30-417e-870e-8d321dcf59a5 service cinder] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:02.401526 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1294/2570] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:02.502308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a688453-6772-4bb2-b854-da98878fa1bf None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:02.512416 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a688453-6772-4bb2-b854-da98878fa1bf None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:02.515934 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a53c565-9e7c-464b-903b-1da0bd90542f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:02.516580 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53c565-9e7c-464b-903b-1da0bd90542f service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:02.516862 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53c565-9e7c-464b-903b-1da0bd90542f service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:02.517169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a53c565-9e7c-464b-903b-1da0bd90542f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:02.576116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53c565-9e7c-464b-903b-1da0bd90542f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:02.577600 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a53c565-9e7c-464b-903b-1da0bd90542f service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:02.587553 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1277/2571] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:05.216214 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41114d33-ba30-417e-870e-8d321dcf59a5 service cinder] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:05.288084 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41114d33-ba30-417e-870e-8d321dcf59a5 service cinder] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:05.292512 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:05.293401 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:05.293705 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:05.293978 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] PATH_INFO: `/v3/users/3685a176c372466b92bbaa9cbacdea8d` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:05.295182 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3685a176c372466b92bbaa9cbacdea8d)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:05.297150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:05.357900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3685a176c372466b92bbaa9cbacdea8d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.366442 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3685a176c372466b92bbaa9cbacdea8d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.367452 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3685a176c372466b92bbaa9cbacdea8d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.374796 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3685a176c372466b92bbaa9cbacdea8d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.389349 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3685a176c372466b92bbaa9cbacdea8d'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.395934 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1295/2572] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:36:05 2022] DELETE /identity/v3/users/3685a176c372466b92bbaa9cbacdea8d => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:36:05.400368 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef1249-4c60-45ae-a1f9-af5f41e7c6b6 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:05.400849 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef1249-4c60-45ae-a1f9-af5f41e7c6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:05.400931 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ef1249-4c60-45ae-a1f9-af5f41e7c6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:05.418674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-85ef1249-4c60-45ae-a1f9-af5f41e7c6b6 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:05.420016 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-85ef1249-4c60-45ae-a1f9-af5f41e7c6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:05.485143 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1278/2573] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:05.495447 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:05.497243 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:05.554534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:05.558779 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f729d883-db89-479d-8a3f-61de44a7036e admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:05.563359 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f7fac-33ad-48ed-8a2d-554ec305a654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:05.564546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f7fac-33ad-48ed-8a2d-554ec305a654 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:05.565019 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f7fac-33ad-48ed-8a2d-554ec305a654 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:05.565481 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688f7fac-33ad-48ed-8a2d-554ec305a654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:05.576143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f7fac-33ad-48ed-8a2d-554ec305a654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:05.577621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688f7fac-33ad-48ed-8a2d-554ec305a654 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:05.590088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1296/2574] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:05.874037 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef1249-4c60-45ae-a1f9-af5f41e7c6b6 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:05.875506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-85ef1249-4c60-45ae-a1f9-af5f41e7c6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:05.889774 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ef1249-4c60-45ae-a1f9-af5f41e7c6b6 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:05.894892 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:05.896406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:05.896406 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:05.896616 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] PATH_INFO: `/v3/projects/3ef830774b5945be9c9618d1d1c683c2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:05.897766 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ef830774b5945be9c9618d1d1c683c2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:05.899848 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:05.919365 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=100 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:05.951719 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ef830774b5945be9c9618d1d1c683c2', 'request_id': 'req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.958688 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ef830774b5945be9c9618d1d1c683c2', 'request_id': 'req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.968741 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ef830774b5945be9c9618d1d1c683c2', 'request_id': 'req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.980979 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] The token cache is being invalidate because project 3ef830774b5945be9c9618d1d1c683c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:36:05.981265 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.981953 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:05.983483 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1279/2575] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:36:05 2022] DELETE /identity/v3/projects/3ef830774b5945be9c9618d1d1c683c2 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:36:05.988061 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f7fac-33ad-48ed-8a2d-554ec305a654 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:05.998341 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688f7fac-33ad-48ed-8a2d-554ec305a654 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:06.059724 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:06.060616 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:06.060904 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:06.061185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] PATH_INFO: `/v3/users/c5d2de7fedf948069cec95ee79752bba` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:06.062444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5d2de7fedf948069cec95ee79752bba)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:06.064378 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:06.117673 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5d2de7fedf948069cec95ee79752bba'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.125050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5d2de7fedf948069cec95ee79752bba'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.125771 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5d2de7fedf948069cec95ee79752bba'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.134516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5d2de7fedf948069cec95ee79752bba'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.147476 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-fa8cbd1a-b4fe-41fa-af93-1a1187cf3663 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5d2de7fedf948069cec95ee79752bba'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.153621 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1297/2576] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:36:05 2022] DELETE /identity/v3/users/c5d2de7fedf948069cec95ee79752bba => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:36:06.439549 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:06.441917 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:06.455787 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9623ee-70a1-4ac2-b520-0a22341f1d4e admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:06.522630 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:06.523444 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:06.523731 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:06.524013 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] PATH_INFO: `/v3/projects/9bf9f5c988944de5a20e72dcff854dfa` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:06.525250 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf9f5c988944de5a20e72dcff854dfa)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:06.527234 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:06.580569 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf9f5c988944de5a20e72dcff854dfa', 'request_id': 'req-847d788c-d98d-4acc-b787-ca1e1ecc879e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.587937 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf9f5c988944de5a20e72dcff854dfa', 'request_id': 'req-847d788c-d98d-4acc-b787-ca1e1ecc879e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.596988 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf9f5c988944de5a20e72dcff854dfa', 'request_id': 'req-847d788c-d98d-4acc-b787-ca1e1ecc879e'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.609994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] The token cache is being invalidate because project 9bf9f5c988944de5a20e72dcff854dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:36:06.610283 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.610935 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-847d788c-d98d-4acc-b787-ca1e1ecc879e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:36:06.613679 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1280/2577] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:36:06 2022] DELETE /identity/v3/projects/9bf9f5c988944de5a20e72dcff854dfa => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:36:06.618659 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34be463-93da-4cda-92a6-c1197d724339 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:06.619159 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34be463-93da-4cda-92a6-c1197d724339 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:06.619280 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34be463-93da-4cda-92a6-c1197d724339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:06.641314 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-c34be463-93da-4cda-92a6-c1197d724339 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:06.642581 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-c34be463-93da-4cda-92a6-c1197d724339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:06.705358 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1298/2578] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:06.715578 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6f6a4-adcf-44b8-be4a-88a0a8eea584 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:06.715965 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6f6a4-adcf-44b8-be4a-88a0a8eea584 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:06.716319 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba6f6a4-adcf-44b8-be4a-88a0a8eea584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:06.737744 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-8ba6f6a4-adcf-44b8-be4a-88a0a8eea584 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:06.739576 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-8ba6f6a4-adcf-44b8-be4a-88a0a8eea584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:06.747269 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1281/2579] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:06.751601 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34be463-93da-4cda-92a6-c1197d724339 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:06.761534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34be463-93da-4cda-92a6-c1197d724339 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:06.766081 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9b50b-a88b-4abe-881f-732e69604a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:06.766745 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9b50b-a88b-4abe-881f-732e69604a0f admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:06.767032 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9b50b-a88b-4abe-881f-732e69604a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:06.767310 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9b50b-a88b-4abe-881f-732e69604a0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:06.768051 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9b50b-a88b-4abe-881f-732e69604a0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:36:06.769172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9b50b-a88b-4abe-881f-732e69604a0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:06.770840 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9b50b-a88b-4abe-881f-732e69604a0f admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:06.778082 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:06.787946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1299/2580] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:36:06.792349 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6f6a4-adcf-44b8-be4a-88a0a8eea584 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:06.802110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba6f6a4-adcf-44b8-be4a-88a0a8eea584 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:06.807240 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d28ff1-e310-4789-9bd3-3a03e9de0cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:06.807911 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d28ff1-e310-4789-9bd3-3a03e9de0cd4 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:06.808197 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d28ff1-e310-4789-9bd3-3a03e9de0cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:06.808477 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d28ff1-e310-4789-9bd3-3a03e9de0cd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:06.809189 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d28ff1-e310-4789-9bd3-3a03e9de0cd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:06.811135 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d28ff1-e310-4789-9bd3-3a03e9de0cd4 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:06.849664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1282/2581] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:36:06 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:36:06.855500 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9b50b-a88b-4abe-881f-732e69604a0f admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:06.871009 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9b50b-a88b-4abe-881f-732e69604a0f admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:06.933388 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfe281-0cf7-487d-94ee-b9064dc81fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:06.934088 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfe281-0cf7-487d-94ee-b9064dc81fce admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:06.934373 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfe281-0cf7-487d-94ee-b9064dc81fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:06.934651 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cfe281-0cf7-487d-94ee-b9064dc81fce admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:06.935352 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfe281-0cf7-487d-94ee-b9064dc81fce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:06.937261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cfe281-0cf7-487d-94ee-b9064dc81fce admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:06.958988 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1300/2582] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:36:06 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:36:06.963067 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d28ff1-e310-4789-9bd3-3a03e9de0cd4 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:06.974413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d28ff1-e310-4789-9bd3-3a03e9de0cd4 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:06.979039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895d315-7f3c-4f4f-bd4d-f0812a8ca60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:06.979706 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895d315-7f3c-4f4f-bd4d-f0812a8ca60f admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:06.979989 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895d315-7f3c-4f4f-bd4d-f0812a8ca60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:06.980262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5895d315-7f3c-4f4f-bd4d-f0812a8ca60f admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:06.980986 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895d315-7f3c-4f4f-bd4d-f0812a8ca60f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:06.982625 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5895d315-7f3c-4f4f-bd4d-f0812a8ca60f admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:06.984095 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:06.996270 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1283/2583] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:36:07.000435 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfe281-0cf7-487d-94ee-b9064dc81fce admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:07.010499 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cfe281-0cf7-487d-94ee-b9064dc81fce admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:07.015134 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f41ed-2578-4c4c-8a71-7c1bcdb2582d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:07.015802 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f41ed-2578-4c4c-8a71-7c1bcdb2582d admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:07.016091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f41ed-2578-4c4c-8a71-7c1bcdb2582d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:07.016374 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f41ed-2578-4c4c-8a71-7c1bcdb2582d admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:07.017119 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f41ed-2578-4c4c-8a71-7c1bcdb2582d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:07.018941 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f41ed-2578-4c4c-8a71-7c1bcdb2582d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:07.030438 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1301/2584] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:36:07.035718 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895d315-7f3c-4f4f-bd4d-f0812a8ca60f admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:07.046077 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5895d315-7f3c-4f4f-bd4d-f0812a8ca60f admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:07.053941 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230e4b0-8111-4976-9405-f633ae6428af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:07.054845 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230e4b0-8111-4976-9405-f633ae6428af admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:07.055205 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230e4b0-8111-4976-9405-f633ae6428af admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:07.055654 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3230e4b0-8111-4976-9405-f633ae6428af admin admin] PATH_INFO: `/v3/projects/a8d960c42bd04476bd6f9b83ae7a2fb6/users/9a8c2a9145ac48fd8ddf2f5f76cc6425/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:07.073120 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230e4b0-8111-4976-9405-f633ae6428af admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8d960c42bd04476bd6f9b83ae7a2fb6, user_id=9a8c2a9145ac48fd8ddf2f5f76cc6425, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:07.077065 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3230e4b0-8111-4976-9405-f633ae6428af admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:07.091645 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1284/2585] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:36:07 2022] PUT /identity/v3/projects/a8d960c42bd04476bd6f9b83ae7a2fb6/users/9a8c2a9145ac48fd8ddf2f5f76cc6425/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:36:07.101106 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d15b2-c5cc-41f6-b5c3-0944de2bf5e9 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:07.101419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d15b2-c5cc-41f6-b5c3-0944de2bf5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:07.101707 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d15b2-c5cc-41f6-b5c3-0944de2bf5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:07.122791 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-339d15b2-c5cc-41f6-b5c3-0944de2bf5e9 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:07.124047 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-339d15b2-c5cc-41f6-b5c3-0944de2bf5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:07.185554 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1302/2586] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:07.196190 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230e4b0-8111-4976-9405-f633ae6428af admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:07.197926 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-3230e4b0-8111-4976-9405-f633ae6428af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:07.257318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3230e4b0-8111-4976-9405-f633ae6428af admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:07.260997 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e552f59-c795-4377-bbe2-c0a04f446667 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:07.261682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e552f59-c795-4377-bbe2-c0a04f446667 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:07.261972 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e552f59-c795-4377-bbe2-c0a04f446667 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:07.262253 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e552f59-c795-4377-bbe2-c0a04f446667 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:07.272582 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e552f59-c795-4377-bbe2-c0a04f446667 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:07.274950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e552f59-c795-4377-bbe2-c0a04f446667 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:07.286964 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1285/2587] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:08.227010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b5b4f-797b-4310-861c-2df37659be9b None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:08.227534 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b5b4f-797b-4310-861c-2df37659be9b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:08.227668 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90b5b4f-797b-4310-861c-2df37659be9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:08.255822 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d90b5b4f-797b-4310-861c-2df37659be9b None None] MFA Rules not processed for user `9a8c2a9145ac48fd8ddf2f5f76cc6425`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:08.257098 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d90b5b4f-797b-4310-861c-2df37659be9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:08.328990 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1303/2588] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:08 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:08.340746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e552f59-c795-4377-bbe2-c0a04f446667 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:08.342233 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2e552f59-c795-4377-bbe2-c0a04f446667 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:08.415569 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e552f59-c795-4377-bbe2-c0a04f446667 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:08.419530 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8aad3-3e9f-4f05-a356-c138ddd6af3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b4f12d08d7084387a1bea7865bf77d0e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:08.420227 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8aad3-3e9f-4f05-a356-c138ddd6af3d service cinder] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:08.420573 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8aad3-3e9f-4f05-a356-c138ddd6af3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:08.420996 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8aad3-3e9f-4f05-a356-c138ddd6af3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:08.430516 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8aad3-3e9f-4f05-a356-c138ddd6af3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:08.431967 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8aad3-3e9f-4f05-a356-c138ddd6af3d service cinder] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:08.442565 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1286/2589] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:09.820211 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812a72-885a-44b5-97d9-8ac5809f4801 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:09.820787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812a72-885a-44b5-97d9-8ac5809f4801 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:09.820787 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e812a72-885a-44b5-97d9-8ac5809f4801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:09.833259 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:09.839336 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-5e812a72-885a-44b5-97d9-8ac5809f4801 None None] MFA Rules not processed for user `9a8c2a9145ac48fd8ddf2f5f76cc6425`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:09.840560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-5e812a72-885a-44b5-97d9-8ac5809f4801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:09.847818 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1304/2590] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:11.013591 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8aad3-3e9f-4f05-a356-c138ddd6af3d service cinder] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:11.015203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-74a8aad3-3e9f-4f05-a356-c138ddd6af3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:11.093914 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8aad3-3e9f-4f05-a356-c138ddd6af3d service cinder] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:11.100203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e97119806774c2482a9886b11ab6f70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:11.101055 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:11.101503 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:11.101905 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:11.104141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:11.117695 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:11.119321 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:11.130711 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1287/2591] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 12 13:36:19.092989 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1007b4a-35eb-4339-bb63-c14f0d19e859 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:19.093626 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1007b4a-35eb-4339-bb63-c14f0d19e859 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:19.094201 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1007b4a-35eb-4339-bb63-c14f0d19e859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:19.135135 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-b1007b4a-35eb-4339-bb63-c14f0d19e859 None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:19.136372 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-b1007b4a-35eb-4339-bb63-c14f0d19e859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:19.206529 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1305/2592] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:36:19 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:19.256622 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:19.258121 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:19.267699 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:19.337536 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9febf1a5-bd1d-4546-acae-bd0eb0d9f7ed service placement] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:19.342493 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4303b65e-7fc8-40ab-877b-9e36b5130770 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:19.343203 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303b65e-7fc8-40ab-877b-9e36b5130770 service swift] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:19.343490 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303b65e-7fc8-40ab-877b-9e36b5130770 service swift] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:19.343770 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4303b65e-7fc8-40ab-877b-9e36b5130770 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:19.353116 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303b65e-7fc8-40ab-877b-9e36b5130770 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:19.354776 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4303b65e-7fc8-40ab-877b-9e36b5130770 service swift] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:19.363918 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1288/2593] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:36:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:36:28.750199 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a8cd-4502-41ce-8192-6e8313ec2cd0 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:28.751704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a8cd-4502-41ce-8192-6e8313ec2cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:28.751704 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d80a8cd-4502-41ce-8192-6e8313ec2cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:28.774674 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-4d80a8cd-4502-41ce-8192-6e8313ec2cd0 None None] MFA Rules not processed for user `9a8c2a9145ac48fd8ddf2f5f76cc6425`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:28.776143 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-4d80a8cd-4502-41ce-8192-6e8313ec2cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:28.803598 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1306/2594] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:29.360137 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=29 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:30.795910 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0c00-42f6-4357-beb8-342d26142a4a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:30.796633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0c00-42f6-4357-beb8-342d26142a4a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:30.796633 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410d0c00-42f6-4357-beb8-342d26142a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:30.821746 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-410d0c00-42f6-4357-beb8-342d26142a4a None None] MFA Rules not processed for user `8aa0b8a0b7ff4eb0b13fb985dc0d710a`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:30.822993 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-410d0c00-42f6-4357-beb8-342d26142a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:30.899413 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1289/2595] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:36:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:36.320055 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a011948-9f7c-477a-890b-5d665a5d88a8 None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.320591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a011948-9f7c-477a-890b-5d665a5d88a8 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.320591 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a011948-9f7c-477a-890b-5d665a5d88a8 None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.322062 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1307/2596] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:36:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:36.329758 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0a821-8683-4bc4-ac9b-2487aeea6a8a None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.330388 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0a821-8683-4bc4-ac9b-2487aeea6a8a None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.331034 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f0a821-8683-4bc4-ac9b-2487aeea6a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.368994 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-87f0a821-8683-4bc4-ac9b-2487aeea6a8a None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:36.370256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-87f0a821-8683-4bc4-ac9b-2487aeea6a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:36.474804 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1290/2597] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:36:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:36.487261 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a011948-9f7c-477a-890b-5d665a5d88a8 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:36.489218 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-8a011948-9f7c-477a-890b-5d665a5d88a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:36.560633 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a011948-9f7c-477a-890b-5d665a5d88a8 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:36.564632 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ba8434-0853-4d9b-9922-b2f7f5bf85dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0ff10db6937b4f2b889c30dd5fad8111', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:36.565353 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba8434-0853-4d9b-9922-b2f7f5bf85dc service glance] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.565736 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba8434-0853-4d9b-9922-b2f7f5bf85dc service glance] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.566014 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ba8434-0853-4d9b-9922-b2f7f5bf85dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.577094 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba8434-0853-4d9b-9922-b2f7f5bf85dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:36.578593 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ba8434-0853-4d9b-9922-b2f7f5bf85dc service glance] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:36.590821 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1308/2598] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:36.690933 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa8bed-eff6-4434-b3d9-d1fcd7c801fd None None] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.691298 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa8bed-eff6-4434-b3d9-d1fcd7c801fd None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.691486 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa8bed-eff6-4434-b3d9-d1fcd7c801fd None None] PATH_INFO: `/` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.693352 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1291/2599] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:36.703095 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003bc46-c05c-4008-9155-34e8bea91c23 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.703636 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003bc46-c05c-4008-9155-34e8bea91c23 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.703943 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f003bc46-c05c-4008-9155-34e8bea91c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.728236 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-f003bc46-c05c-4008-9155-34e8bea91c23 None None] MFA Rules not processed for user `83837e4f8f9b4e538658876cfb56db16`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:36.729533 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-f003bc46-c05c-4008-9155-34e8bea91c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:36.736503 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1309/2600] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:36:36 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:36.750023 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa8bed-eff6-4434-b3d9-d1fcd7c801fd None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:36.751838 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a5aa8bed-eff6-4434-b3d9-d1fcd7c801fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:36.773272 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa8bed-eff6-4434-b3d9-d1fcd7c801fd None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:36.777545 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16124593-0691-438b-aca5-5817589d4bed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:36.778308 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16124593-0691-438b-aca5-5817589d4bed service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.778598 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16124593-0691-438b-aca5-5817589d4bed service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.778871 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16124593-0691-438b-aca5-5817589d4bed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.788568 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16124593-0691-438b-aca5-5817589d4bed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:36.790080 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16124593-0691-438b-aca5-5817589d4bed service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:36.800207 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1292/2601] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:36.902556 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2a0fd-8ed7-45b6-842d-5c2511fe1c4b None None] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.902916 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2a0fd-8ed7-45b6-842d-5c2511fe1c4b None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.903385 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a2a0fd-8ed7-45b6-842d-5c2511fe1c4b None None] PATH_INFO: `/` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.905160 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1310/2602] 10.222.0.23 () {54 vars in 921 bytes} [Tue Jul 12 13:36:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 12 13:36:36.912243 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f9c2-9702-4f13-9896-ebe8e39a1a35 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.912586 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f9c2-9702-4f13-9896-ebe8e39a1a35 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.912808 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643f9c2-9702-4f13-9896-ebe8e39a1a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.935595 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-2643f9c2-9702-4f13-9896-ebe8e39a1a35 None None] MFA Rules not processed for user `1cc0050f14664630a36814ca670f124f`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:36.936875 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-2643f9c2-9702-4f13-9896-ebe8e39a1a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:36.948169 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1293/2603] 10.222.0.23 () {58 vars in 1018 bytes} [Tue Jul 12 13:36:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:36.960423 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2a0fd-8ed7-45b6-842d-5c2511fe1c4b None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:36.973946 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a2a0fd-8ed7-45b6-842d-5c2511fe1c4b None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:36.977546 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ab087-7bb2-4001-8c31-a09bb2984589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:36.978194 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ab087-7bb2-4001-8c31-a09bb2984589 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:36.978483 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ab087-7bb2-4001-8c31-a09bb2984589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:36.978766 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2ab087-7bb2-4001-8c31-a09bb2984589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:36.992900 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ab087-7bb2-4001-8c31-a09bb2984589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:36.994731 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2ab087-7bb2-4001-8c31-a09bb2984589 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:37.013115 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1311/2604] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:36:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:36:39.468287 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb89cd1-a448-4831-b593-f326d2bea893 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:39.468830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb89cd1-a448-4831-b593-f326d2bea893 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:39.468830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb89cd1-a448-4831-b593-f326d2bea893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:39.488681 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-7cb89cd1-a448-4831-b593-f326d2bea893 None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:36:39.490122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-7cb89cd1-a448-4831-b593-f326d2bea893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:36:39.497514 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1294/2605] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:36:39 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:36:39.510322 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ab087-7bb2-4001-8c31-a09bb2984589 service neutron] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:36:39.522281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2ab087-7bb2-4001-8c31-a09bb2984589 service neutron] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:36:39.527354 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6dca8-6709-4e2e-8b43-24467ccff722 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:36:39.527997 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6dca8-6709-4e2e-8b43-24467ccff722 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:36:39.528283 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6dca8-6709-4e2e-8b43-24467ccff722 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:36:39.528560 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6dca8-6709-4e2e-8b43-24467ccff722 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:36:39.539589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6dca8-6709-4e2e-8b43-24467ccff722 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:36:39.541180 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6dca8-6709-4e2e-8b43-24467ccff722 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:36:39.553216 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1312/2606] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:36:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:36:49.482309 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=77 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:36:49.547111 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=76 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:37:00.885318 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90498c74-3600-4ba4-9791-b4566783368b None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:00.885921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90498c74-3600-4ba4-9791-b4566783368b None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:00.885921 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90498c74-3600-4ba4-9791-b4566783368b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:00.913830 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-90498c74-3600-4ba4-9791-b4566783368b None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:37:00.915598 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-90498c74-3600-4ba4-9791-b4566783368b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:37:00.924734 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1295/2607] 10.222.0.23 () {58 vars in 967 bytes} [Tue Jul 12 13:37:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:37:00.940589 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6dca8-6709-4e2e-8b43-24467ccff722 service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:00.955743 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6dca8-6709-4e2e-8b43-24467ccff722 service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:00.964615 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a781e-b889-46b6-a9ab-5d7613a30845 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:00.966008 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a781e-b889-46b6-a9ab-5d7613a30845 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:00.966181 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a781e-b889-46b6-a9ab-5d7613a30845 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:00.966477 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743a781e-b889-46b6-a9ab-5d7613a30845 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:00.980043 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a781e-b889-46b6-a9ab-5d7613a30845 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:00.981690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743a781e-b889-46b6-a9ab-5d7613a30845 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:00.993013 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1313/2608] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:37:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:37:10.409950 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d87869-a332-438d-8a1a-c1b9693b1f87 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:10.410482 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d87869-a332-438d-8a1a-c1b9693b1f87 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:10.410575 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d87869-a332-438d-8a1a-c1b9693b1f87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:10.433840 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-a2d87869-a332-438d-8a1a-c1b9693b1f87 None None] MFA Rules not processed for user `b26fa3bc9c0d4e22a8abc5a4d98c9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:37:10.435153 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-a2d87869-a332-438d-8a1a-c1b9693b1f87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:37:10.444690 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1296/2609] 10.222.0.23 () {58 vars in 1009 bytes} [Tue Jul 12 13:37:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:37:10.457513 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a781e-b889-46b6-a9ab-5d7613a30845 service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:10.469494 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743a781e-b889-46b6-a9ab-5d7613a30845 service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:10.475150 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af113a-6e76-421a-bc1b-ec9bd51afc28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b69264fe173045e08c282171cd7370af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:10.475820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af113a-6e76-421a-bc1b-ec9bd51afc28 service swift] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:10.476109 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af113a-6e76-421a-bc1b-ec9bd51afc28 service swift] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:10.476390 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af113a-6e76-421a-bc1b-ec9bd51afc28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:10.486721 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af113a-6e76-421a-bc1b-ec9bd51afc28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:10.488333 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71af113a-6e76-421a-bc1b-ec9bd51afc28 service swift] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:10.499015 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1314/2610] 10.222.0.23 () {60 vars in 1384 bytes} [Tue Jul 12 13:37:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 12 13:37:16.930257 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d87869-a332-438d-8a1a-c1b9693b1f87 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:16.943828 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d87869-a332-438d-8a1a-c1b9693b1f87 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:16.948327 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d5f41-2041-4662-9c46-c6420d9a13b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:16.949133 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d5f41-2041-4662-9c46-c6420d9a13b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:16.949527 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d5f41-2041-4662-9c46-c6420d9a13b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:16.949885 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3d5f41-2041-4662-9c46-c6420d9a13b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:16.951857 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-4a3d5f41-2041-4662-9c46-c6420d9a13b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:37:16.969122 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d5f41-2041-4662-9c46-c6420d9a13b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:16.969463 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3d5f41-2041-4662-9c46-c6420d9a13b6 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:16.981353 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1297/2611] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:37:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:37:20.494010 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=12 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:37:21.117419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af113a-6e76-421a-bc1b-ec9bd51afc28 service swift] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:21.118895 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-71af113a-6e76-421a-bc1b-ec9bd51afc28 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:37:21.137820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af113a-6e76-421a-bc1b-ec9bd51afc28 service swift] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:21.152770 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:21.156857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:21.156857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:21.156857 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] PATH_INFO: `/v3/users/c4b710bbdf0c4d58b4f3ee7ef7410c0a` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:21.185858 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4b710bbdf0c4d58b4f3ee7ef7410c0a)` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:21.190611 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:21.285530 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4b710bbdf0c4d58b4f3ee7ef7410c0a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.301952 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4b710bbdf0c4d58b4f3ee7ef7410c0a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.303124 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4b710bbdf0c4d58b4f3ee7ef7410c0a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.316849 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4b710bbdf0c4d58b4f3ee7ef7410c0a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.339227 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.notifications [None req-16c7ae09-84e9-45ff-8e76-59b43353a76d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4b710bbdf0c4d58b4f3ee7ef7410c0a'} {{(pid=80288) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.349079 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1315/2612] 10.222.0.23 () {58 vars in 1223 bytes} [Tue Jul 12 13:37:21 2022] DELETE /identity/v3/users/c4b710bbdf0c4d58b4f3ee7ef7410c0a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 12 13:37:21.602052 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d5f41-2041-4662-9c46-c6420d9a13b6 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:21.618620 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3d5f41-2041-4662-9c46-c6420d9a13b6 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:21.748284 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:21.749484 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:21.750045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:21.750450 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] PATH_INFO: `/v3/projects/ab8fca4ab6ed466a864c5551fea5f7ff` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:21.775238 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab8fca4ab6ed466a864c5551fea5f7ff)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:21.779345 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:21.857074 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab8fca4ab6ed466a864c5551fea5f7ff', 'request_id': 'req-d036c505-d188-498c-9189-329dbf262de5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.869204 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab8fca4ab6ed466a864c5551fea5f7ff', 'request_id': 'req-d036c505-d188-498c-9189-329dbf262de5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.880039 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab8fca4ab6ed466a864c5551fea5f7ff', 'request_id': 'req-d036c505-d188-498c-9189-329dbf262de5'} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.913601 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] The token cache is being invalidate because project ab8fca4ab6ed466a864c5551fea5f7ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80287) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 12 13:37:21.914400 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.915263 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.notifications [None req-d036c505-d188-498c-9189-329dbf262de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80287) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 12 13:37:21.917022 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1298/2613] 10.222.0.23 () {58 vars in 1232 bytes} [Tue Jul 12 13:37:21 2022] DELETE /identity/v3/projects/ab8fca4ab6ed466a864c5551fea5f7ff => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:37:21.922132 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cceea-bf09-4893-b5bb-97e083b9a4d5 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:21.923151 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cceea-bf09-4893-b5bb-97e083b9a4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:21.924408 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8cceea-bf09-4893-b5bb-97e083b9a4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:21.953147 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-ad8cceea-bf09-4893-b5bb-97e083b9a4d5 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:37:21.954947 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-ad8cceea-bf09-4893-b5bb-97e083b9a4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:37:22.056408 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0589fad-bd0f-463b-a01d-b31508873636 None None] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:22.057090 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0589fad-bd0f-463b-a01d-b31508873636 None None] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:22.057563 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0589fad-bd0f-463b-a01d-b31508873636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:22.060382 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1316/2614] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:37:22.082362 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.auth.core [None req-b0589fad-bd0f-463b-a01d-b31508873636 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80287) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:37:22.085211 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-b0589fad-bd0f-463b-a01d-b31508873636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:37:22.094050 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1299/2615] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 12 13:37:22.101097 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cceea-bf09-4893-b5bb-97e083b9a4d5 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:22.114467 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8cceea-bf09-4893-b5bb-97e083b9a4d5 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:22.123644 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea137c1-4819-4c72-b2cf-1397cd260c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:22.124444 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea137c1-4819-4c72-b2cf-1397cd260c79 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:22.124834 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea137c1-4819-4c72-b2cf-1397cd260c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:22.125228 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea137c1-4819-4c72-b2cf-1397cd260c79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:22.126091 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea137c1-4819-4c72-b2cf-1397cd260c79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80288) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 12 13:37:22.127284 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea137c1-4819-4c72-b2cf-1397cd260c79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:22.129169 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea137c1-4819-4c72-b2cf-1397cd260c79 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:22.146050 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1317/2616] 10.222.0.23 () {58 vars in 1152 bytes} [Tue Jul 12 13:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 12 13:37:22.150208 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0589fad-bd0f-463b-a01d-b31508873636 None None] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:22.161716 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0589fad-bd0f-463b-a01d-b31508873636 None None] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:22.168256 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0b4e4-5ed6-4cff-9a05-6a9678a6cbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:22.169237 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0b4e4-5ed6-4cff-9a05-6a9678a6cbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:22.169760 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0b4e4-5ed6-4cff-9a05-6a9678a6cbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:22.170069 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c0b4e4-5ed6-4cff-9a05-6a9678a6cbd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:22.170796 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0b4e4-5ed6-4cff-9a05-6a9678a6cbd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:22.172932 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c0b4e4-5ed6-4cff-9a05-6a9678a6cbd6 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:22.243433 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea137c1-4819-4c72-b2cf-1397cd260c79 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:22.243964 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1300/2617] 10.222.0.23 () {60 vars in 1152 bytes} [Tue Jul 12 13:37:22 2022] POST /identity/v3/projects => generated 385 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 12 13:37:22.264983 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea137c1-4819-4c72-b2cf-1397cd260c79 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:22.344039 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:37:22.361691 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2dd22-74bc-47fc-a7c9-32d3197889d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:22.362652 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2dd22-74bc-47fc-a7c9-32d3197889d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:22.363122 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2dd22-74bc-47fc-a7c9-32d3197889d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:22.363585 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2dd22-74bc-47fc-a7c9-32d3197889d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:22.364295 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2dd22-74bc-47fc-a7c9-32d3197889d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:22.366660 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a2dd22-74bc-47fc-a7c9-32d3197889d7 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:22.391898 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1318/2618] 10.222.0.23 () {60 vars in 1143 bytes} [Tue Jul 12 13:37:22 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 12 13:37:22.396262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0b4e4-5ed6-4cff-9a05-6a9678a6cbd6 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:22.432431 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c0b4e4-5ed6-4cff-9a05-6a9678a6cbd6 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:22.441664 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf2912-eaae-4d55-9ce3-ceb7ff799852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:22.442811 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf2912-eaae-4d55-9ce3-ceb7ff799852 admin admin] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:22.443506 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf2912-eaae-4d55-9ce3-ceb7ff799852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:22.443966 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bf2912-eaae-4d55-9ce3-ceb7ff799852 admin admin] PATH_INFO: `/v3/roles` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:22.445262 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf2912-eaae-4d55-9ce3-ceb7ff799852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:22.448541 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bf2912-eaae-4d55-9ce3-ceb7ff799852 admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:22.481210 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1301/2619] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:37:22.483714 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2dd22-74bc-47fc-a7c9-32d3197889d7 admin admin] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:22.515820 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2dd22-74bc-47fc-a7c9-32d3197889d7 admin admin] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:22.527334 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9ac076-fc09-4574-87ef-60ed2195aba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:22.528077 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ac076-fc09-4574-87ef-60ed2195aba9 admin admin] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:22.528345 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ac076-fc09-4574-87ef-60ed2195aba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:22.528628 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9ac076-fc09-4574-87ef-60ed2195aba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:22.529397 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ac076-fc09-4574-87ef-60ed2195aba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:22.531057 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9ac076-fc09-4574-87ef-60ed2195aba9 admin admin] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:22.546327 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1319/2620] 10.222.0.23 () {58 vars in 1121 bytes} [Tue Jul 12 13:37:22 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 12 13:37:22.547589 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf2912-eaae-4d55-9ce3-ceb7ff799852 admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:22.558527 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bf2912-eaae-4d55-9ce3-ceb7ff799852 admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:22.566459 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9202fab-47fb-47f5-a3d2-bbcf6542beac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '417a441bd00d4df89cd837724e786c1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '65457e2fc15c4d94825d2ac5ee397f82', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:22.567394 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9202fab-47fb-47f5-a3d2-bbcf6542beac admin admin] REQUEST_METHOD: `PUT` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:22.567680 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9202fab-47fb-47f5-a3d2-bbcf6542beac admin admin] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:22.568328 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9202fab-47fb-47f5-a3d2-bbcf6542beac admin admin] PATH_INFO: `/v3/projects/52adc9255289414bab3089bf72284758/users/efb97b4d9c654f6898ecee7fc217ce8b/roles/5a1c0efbb0f14248bf3a2f523478eef2` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:22.572618 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:37:22.590141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9202fab-47fb-47f5-a3d2-bbcf6542beac admin admin] RBAC: Authorizing `identity:create_grant(project_id=52adc9255289414bab3089bf72284758, user_id=efb97b4d9c654f6898ecee7fc217ce8b, role_id=5a1c0efbb0f14248bf3a2f523478eef2)` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:22.593045 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9202fab-47fb-47f5-a3d2-bbcf6542beac admin admin] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:22.617474 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1302/2621] 10.222.0.23 () {60 vars in 1482 bytes} [Tue Jul 12 13:37:22 2022] PUT /identity/v3/projects/52adc9255289414bab3089bf72284758/users/efb97b4d9c654f6898ecee7fc217ce8b/roles/5a1c0efbb0f14248bf3a2f523478eef2 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 12 13:37:22.619281 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1ab2-0cdc-4282-bae0-6d43de775277 None None] REQUEST_METHOD: `POST` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:22.619577 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1ab2-0cdc-4282-bae0-6d43de775277 None None] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:22.619860 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12a1ab2-0cdc-4282-bae0-6d43de775277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:22.639172 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.auth.core [None req-d12a1ab2-0cdc-4282-bae0-6d43de775277 None None] MFA Rules not processed for user `417a441bd00d4df89cd837724e786c1e`. Rule list: `[]` (Enabled: `True`). {{(pid=80288) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 12 13:37:22.640419 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d12a1ab2-0cdc-4282-bae0-6d43de775277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:37:22.717146 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1320/2622] 10.222.0.23 () {58 vars in 957 bytes} [Tue Jul 12 13:37:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 12 13:37:22.727835 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9202fab-47fb-47f5-a3d2-bbcf6542beac admin admin] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:37:22.729762 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-d9202fab-47fb-47f5-a3d2-bbcf6542beac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:37:22.797110 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9202fab-47fb-47f5-a3d2-bbcf6542beac admin admin] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:37:22.802771 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4a7f8-60d2-4ac5-8e66-faff2e9b0019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:37:22.803458 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4a7f8-60d2-4ac5-8e66-faff2e9b0019 service neutron] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:37:22.803747 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4a7f8-60d2-4ac5-8e66-faff2e9b0019 service neutron] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:37:22.804033 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e4a7f8-60d2-4ac5-8e66-faff2e9b0019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:37:22.814674 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4a7f8-60d2-4ac5-8e66-faff2e9b0019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:37:22.816216 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e4a7f8-60d2-4ac5-8e66-faff2e9b0019 service neutron] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:37:22.831178 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1303/2623] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:37:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:37:32.711962 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=39,keystone:INSERT=3 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:37:32.824228 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:38:22.896665 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a1ab2-0cdc-4282-bae0-6d43de775277 None None] Authenticating user token {{(pid=80288) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:38:22.898185 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-d12a1ab2-0cdc-4282-bae0-6d43de775277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:22.910215 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12a1ab2-0cdc-4282-bae0-6d43de775277 None None] Validating token access rules against request {{(pid=80288) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:38:22.914482 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e692067d-ad8d-4ade-8e66-8c8c3c31a7f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '83837e4f8f9b4e538658876cfb56db16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80288) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:38:22.915123 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692067d-ad8d-4ade-8e66-8c8c3c31a7f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:38:22.915410 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692067d-ad8d-4ade-8e66-8c8c3c31a7f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:38:22.915690 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e692067d-ad8d-4ade-8e66-8c8c3c31a7f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80288) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:38:22.917489 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.fernet_utils [None req-e692067d-ad8d-4ade-8e66-8c8c3c31a7f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80288) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:38:22.926891 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692067d-ad8d-4ade-8e66-8c8c3c31a7f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:38:22.928317 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e692067d-ad8d-4ade-8e66-8c8c3c31a7f7 service neutron] RBAC: Authorization granted {{(pid=80288) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:38:22.936516 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: [pid: 80288|app: 0|req: 1321/2624] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:38:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:38:32.932735 dv-cinder-820027-10-iscsi devstack@keystone.service[80288]: DEBUG dbcounter [-] [80288] Writing DB stats keystone:SELECT=6 {{(pid=80288) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 12 13:41:17.509141 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4a7f8-60d2-4ac5-8e66-faff2e9b0019 service neutron] Authenticating user token {{(pid=80287) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 12 13:41:17.510607 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-42e4a7f8-60d2-4ac5-8e66-faff2e9b0019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:17.581728 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e4a7f8-60d2-4ac5-8e66-faff2e9b0019 service neutron] Validating token access rules against request {{(pid=80287) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 12 13:41:17.586704 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a92f08-6745-4a12-9a31-36df70531ee4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2e97119806774c2482a9886b11ab6f70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e60ff54d1d814805b912118b9b02cc28', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80287) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 12 13:41:17.587351 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a92f08-6745-4a12-9a31-36df70531ee4 service placement] REQUEST_METHOD: `GET` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 12 13:41:17.587638 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a92f08-6745-4a12-9a31-36df70531ee4 service placement] SCRIPT_NAME: `/identity` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 12 13:41:17.587916 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a92f08-6745-4a12-9a31-36df70531ee4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80287) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 12 13:41:17.589727 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.fernet_utils [None req-30a92f08-6745-4a12-9a31-36df70531ee4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80287) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 12 13:41:17.601682 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a92f08-6745-4a12-9a31-36df70531ee4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 12 13:41:17.603270 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a92f08-6745-4a12-9a31-36df70531ee4 service placement] RBAC: Authorization granted {{(pid=80287) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 12 13:41:17.613182 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: [pid: 80287|app: 0|req: 1304/2625] 10.222.0.23 () {60 vars in 1365 bytes} [Tue Jul 12 13:41:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 12 13:41:27.608998 dv-cinder-820027-10-iscsi devstack@keystone.service[80287]: DEBUG dbcounter [-] [80287] Writing DB stats keystone:SELECT=32 {{(pid=80287) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}